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ABSTBACT ^ 

Because of the^constant increase in knowledge, 
students can no longer be^^ught all thej need to learn*. The focus of 
education nust therefore shift toward assisting thea to. learn hov to 
learn, hov to be independent, and hov to equip thevselves to cope 
vith an ever changing environment • One of the nine papers presented 
describes a vethod bj vhich teachers can facilitate learning* Another 
urges that teachers should give up their traditional role as 
purveyors of information and instead help students to define the 
objective of each learning experience and identify the methods of 
learning they must adopt* Teachers must also appraise; each student's 
progress tovard the educational goals set, and this raises the 
question of examinations, vhich is dealt vith in another paper* There 
is also a discussion of the appraisal of the teachers themselves in 
the light of hov they are perceived by the students, of the process 
of teaching, and of the end products of that teaching process* Other 
subjects covered included the teaching of large groups, the dynamics 
^of learning groups, and an assessment of vhat has been achieved by 
educational science in a specific university medical' school* (HJH} 
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PREFACE 



The importance of educational science in improving both the 
efficiency and the effectiveness of educational programmes for the health 
professions can no longer be seriously questioned. Through a variety 
of activities the World Health Organization has attempted to promote 
the incorporation of this new discipline into the programme planning 
and implementation efforts of health professions schools. As a result 
of the worldwide WHO teacher training programme for the health 
professions there is a steadily increasing pool of teachers who are more 
knowledgeable about the principles uriderlying the educational process, 
and skilled in their application. 

As a WHO Collaborating Institute the Center for Educational 
Development at the University of Illinois College oj Medicine has 
contributed to this work, inter aUa, by preparing reports that sum- 
marize significant ideas from the vast field of educational science in a 
y^ay that illustrates their applicability to health professions programmes. 
The first group of these papers was published in 1973.^ The second 
series is offered here as a companion to that earUer volume All but 
one of the papers included in this volume were prepared by staff 
members of the Center for Educational Development. 



^Dtv0lopmtnt of educational protrammes for th$ htdUh Professions, Geneva, World 
Health Organization* 1973 (JPubUe HtaUh Fapers, No. 52). 
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CURRICULUM THEORY AND 
CURRICULUM PRACTICE 



Allan W. WaUdien^ 



To many teachers and administrators, curriculum ihtoqf may seem 
far removed from the practical, day-to-day operation of medical 
education. The purpose of this paper is to advance the view that 
curriculum theory is altogether integral to the daily practice of educa^ 
ti(m, and to persuade the reader that the judgement of a school's 
quality is in large part a reflection of that school's success in applying 
curriculum theory. 

Obviously a curriculum theory and a curriculum theorist deal with 
curriculum* The words ""theory" and ""theorist'' have no hidden 
meanings and their definitions are universally accepted but the word 
"curriculum" does not enjoy the same acceptance. A curriculum is 
many things to different pcofde. Generally it is regarded as a programme 
of instruction for an educational institution. To students, curriculum is 
usually an array of required or elective courses that must be taken in 
order to graduate. To teachers, curriculum may be a sequence of 
'"packages of knowledge" to be ""tauglit" in certtun ways at certain 
times by subject specialists. To administrators, curriculum is a way of 
orgamidng and departmentalizing a school into manageable administra^ 
tive units. To society, curriculum often represents a time continuum of 
""school activities" that have been dcdgned to prepare students for future 
rdes in the community. Additional^, curriculum can be thouglit of as 
the content for instruction; the standards for student learning; a 
programme for processing students; a tradition for the school to follow; 
a set of guidelines to serve society's needs; or a primary vehicle for 
change in education. All of these perceptions are appropriate and, 
indeed, necessary for full tmderstanding of curriculum theory and its 
translation into practice. With curriculum serving so many functions, it 
is obvious that theory must draw upon all of these elements and 
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synthesize them into a meaningful plan that considers both process and 
product of education* 

A widespread, but overly simplistic, view suggests that the ultimate 
criterion for measuring curriculum in medical education is the quality 
of health scrsdce provided by its graduates. If health care is less than 
adequate — and it is always inadequate in some way — it is predictable 
that someone will propose some curricular change that in his view Mdll 
correct the purported defidency. The critic may be a member of the 
faculty who feels that too many people are dying of heart disease and 
demands extra hours of instruction in cardiac transplantation and its 
immunological detmninants; he may be a practitioner who feels there 
are too many specialists and proposes a course of study that will 
prepare more generalists; he may be a patient who complains that 
doctors charge to much and suggests a practical course in medical 
economics; he may be a taxpayer who thiidcs that it costs too much to 
train ph^cians and itemands that the programme of study be shortened; 
or he may be a politician who is outraged that physicians congregate in 
urban settings and insists upon preparing more doctors to practise 
rural medicine* All these critics have curriculum solutions, but few 
recognize that curriculum is only one of the variables to which attention 
must be directed if serious problems of health service deliveiy are to 
be solved 

Because curriculum is important, even though not the fx>le deter- 
minant of outcome, it is all the more critical that faculty members have 
some solid theoretical framework against which to judge the potentid 
usefulness of the progranmie they offer* 



CURRICULUM MODELS 

One model of curriculum development that seems to offer both simple 
and rational guidelines is illustrated in Fig. L 

The linear logic of this model suggests a termination — Le., the 
creation of ^^curriculum plans" marks the end^point of the planning 
process* Administratively this model seems to make sense: it has a data 
source («&terminants); it identifies the plamming personnel; it clarifies 
the realm of their decision-making; and the process culminates in a 
product (plans) that specifies the programm i of instruction for a schod; 
then planning ?tops and teaching be^* It i \ unfortunate that this over* 
^pUfication too often represents the realitj of practice, for to say that 
curriculum planmng stops when instruction b ?gins is like suggesting that 
learning stops when the class bell ends s^n anatomy lecture or a 
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FIQ. 1. CURRICULUM MODEL— tiNEAR FORM 



Curriculum Curriculum Curriculum Curriculum 

determinants planners decisions plans 

\/ \/ \/ 

GUIDE WHO MAKE WHICH RESULT IN 

WHO40$2$ 



pathology lalxNratory sessioiL Curriculum, like learning, should be a 
dynamic {Mrocess that is responuve to the needs of learners and of 
society* If a curriculum prccess must truly be accountable to those it 
serves then another model is required 

Instead of fdlowing a linear pathway the process should be looped 
to avoid any termination, to assure the dynamism of curriculum 
implementation and to provide a measure of accountability that is 
missing in the linear model It is illustrated schematically in I^g. 2 as a 
closed circle, but the intention is to suggest a never-ending spiral* 



FIQ. Z CURRICULUM MODEL-CIRCULAR FORM 

CURRICULUM 
DETEBMINANTS 

which 

should reflect which guide 



CURRICULUM CURRICULUM 
PLANS PLANNERS 



which 

result in. who make 



CURRICULUM 
DECISIONS wm>4osi$ 



The four parts of this ^^accountable** model (determinants, fdanners, 
decisions, and plans) are linked by qualifying verbs that integrate 
curriculum product with curriculum processes and provide a rational 
framework against which to assess the adequacy of a programme. 

Ml 11 n 
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CimiUCUUm IHnBMilKANIS 

Curricotam determiiuuitt are tbote facixn that estaUith the need for 
and daicribe die aoope of a pcogcanmie* In dila modd deteiininaiila 
inidale d» pcooesa (gi^ die pUno^ 

for meamring die effecdveaeaa cl die plana. Tbe importanoe of iden- 
tifying die dcitenninantt and cl considering die implicatkms of eadi for 
die planning pfxicesa cannot be overstated* 

Generally, four Qfpes of cturiculom determinants can he identified: 
(1) academic; (2) social; 0) economic; and (4) poUticaL These 
catefories are neitber all-indimve nor mutually exdukve; however, diqr 
are fsnerally useful. 

Aakkmk determinants are probably the most significant influences 
on contemporary curric^um planning and in many instances .may be 
virtually the sole determinant Each disdpline has an organized bo^ of 
knowl^ge diat is championed by a vigorous group ci faculty 
protagonists who demand adequate time and opportunity to teadi lAM 
diey have learned Those academicians who represent die bi o medi c al 
sciences are sometimes countered by odier academic forces that reflect 
a different set of concerns, and spedfically concern for the nature of 
the student body u well as the proceu of learmng. The content 
specialists may pre» for a core cuiriculum to serve what di^ stc zz 
the similar needs of a variety of health profestionals in training; the 
process specialists emphasize the diflerences among these studenti, in 
academic preparation, intellectual calibre, and ultimate goab and plead 
for more individuaUy taQored curricula for eadi group* Th^ furdier 
cite the persuadve evidence that individuals learn differently, and thus 
make point for individualized opportunities for learning throo^ 
multifde tracks and active invdvement in learning rather dun passive 
absorpticii of information* When these and odier prindfAes of learning 
are cast against widespread and traditicmal teaching practices, turmoil 
and cofiflict are ine^t^e* But these forces are an important ingredient 
of the icademk determisiants of curriculum that must not be overlooked 
by curriculum planners* 

Social needs should also influence curriculum development and there 
should be overt acknowledgement that such needs will almost 
hievitaUy vary from nation to nation, or even among groups vrithin a 
ttng^e country* In a devdc^g country an ovenrtielming need for 
basic health measures may dictate a auticulum designed to prepare 
profiesdonals primarily for public healUi practice, while hi a higjhly 
industrialized country widi a well-established and comprehensive h^ldi 
service system diere may be for greater need for curricula that prepare 
young neofrfe for hi^y specialized areas of medical diagnosis and 
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Oierapy . lliere need be no invidioiis comparisons of the two approaches; 
one is not good and the odierbad. Eadi is merdy a reflection of sodal 
need diat must be considered in sound curriculum idanning. 

Economic detnminants should influence bofli die nature of a 
curriculum ifor phyridans and die nature ci die natkm's dedsions about 
die kinds of health professionals to be tramed in a country widi a 
physidan/padent rado <^ 1 : IS 000, it seems only rational to plan a 
curriculum that prq>ares physicians for the vile of managers of healdi 
resources rather than as direct purveyon of individual health serwes. 
An institution with a budget of $3000 per student par year should 
consdously plan a very different kind of curriculum tlum one with a 
budget of $ 10 000 per student per year, not merely leave that eccmomic 
factor to be dealt with after the curriculum pianning has been com- 
pleted Feasibility as wefl as desiralrility should concern plannm, fen 
otherwise their - work may lead only to frustraticm when die 
imfdementation of unrealistic 0ans becomes clearly imposdble. 

P<Aiticd determinants of curriculum camK>ty and should not, be 
ignored In a developing nation the contribution to national pride 
made by a medical school that takes as a modd ibt most advanced 
institution in the developed worid may be as agnificant an influence 
upon curriculum jdaoning as the political need in a devdoped country 
to establish programmes that will in^ease the production of physicians 
and shorten the period required to train them. In ndther ^tuation 
should the political needs alone determine the outcome, but at the 
same time it is necessary to deal with them in a forthrigjit foshion. 

When considering the four general categories of curriculum 
determinants, planners will expect to encounter conflict — conflict 
between determinants, conflict between theory and practice, and conflict 
among planning priorities. The anxieties and frustrations that result 
are inherent in and integral to a dynamic and sound plaiming process. 



CURRICULUM PLANNERS 

Obviously, a medical school curriculum cannot be designed by a 
very large group, but those who are immediately affected should 
certainly be represented in any planning committee. Idealty the planners 
would include faculty, administrators, and students who will first 
examine and evaluate the curriculum detemimants in their specific 
setting. 

To be effective, this group must become fully involved in and com- 
mitted to the planning activity. They must be willing to put a»de past 
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practice, present power, personal preferences, and differences in 
pers(Hiality as well as philosophy* Ihey must be prepared to confront 
the conflicts rationally and to deal with each in turn, for otherwise 
pikmties will be very difficult to establish* There should be no dis- 
agreement that cannot be resolved by this group; if there is, then either 
the area of conflict is not within thdr dedsion^maldng power, or the 
group does not have the necessary resources for effective educational 
planning. 



In the past, curriculum planners have generally made organizational 
deci^ons. Ihey have manipulated Uocks of subject-matter or content 
(courses in anatraiy, phyaology, surgery, or interdisdidinary offerings 
in organ-'System packages) or academic time units (terms, semesters, 
clerkship periods, laboratory hours). But these Idnds of ded^ons are 
merely tactical; those required in the "accountable curriculum modd** 
must deal with the level of profesdonal competence that must be 
achieved in order to qualify for graduation from the pr(^gramme or 
for the practice of medidne. The curriculum dedsions demanded here 
will stress performance, not subject organization or assignment of 
time. An illustration of sudi a formulation is gjiven in Fig. 3. 

The process of curriculum development employed by the planners of 
this programme began with identification of the areas in which 
competence should have been acquired fay medical students by the end 
of a medical school progranure. Once agreement had been reached on 
this core, ^)ecific learning objectives were delineated, with the 
organization of dusters of learning activities through which these 
objectives mi^t be achieved There followed a set of instructional 
dedsions througli which such a curriculum would be implemented: the 
specifics of who would do what and when, i.e., the curriculum plan. 

Such curriculum decisions as these focus on the learner, not the 
teacher, and the resulting thrust of instruction is designed to facilitate 
specific learnings that have been identified in terms of performance and 
competence, not courses and content A medical student embarldng 
upon such a programme can shift his energies from enga^ng in feats 
of memory to acquiring the competence that medical practice requires. 
He can become activdy involved in meaningful learning and take much 
of the responsibility for it And since demonstration of defined learning 
rather than pas^g through a specified set of instructional experiences 
is the intent of such a programme, students, &culty, and administration 
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RQ. 3. FORMUUTION OF ACTION SKIUS IN A CURRICULUM* 



Action tkOto 

At the time of graduation fn>m medical school it Is unrealistic to 
expect all students to have acquired a level of professional competence 
sufficient to justify Indep en d e tH management of any significant number 
of health prot>lems. There are some areas, however. In which this 
competence should have been achieved, and these are listed belcw. 
in addition, any student should be able to formulate a general plan of 
management for ttie disease processes he is expected to recognize 
(and which are listed in an earlier section of this document). He should 
also be prepared to outline a diagnostio approach to prominent 
presenting symptoms and signs not Immediately recognizable as specific 
diseases. 

1. bidependenl nMnagement 

— Routine screening procedure for adults, adolescents, dilldren. 

Intents 

— Pre-marital and marital counselling. Including contraceptiva advice 

— Normal pregnancy and delivery 

— Emergency care, acute traumatic Injury 

— Emergency care, cardiac arrest 

— Emergency care, acute poisoning (aspirin, lye, barbiturate, hydro- 

cartx>ns, inescaline) 
— Anterior nasal haemorrhage 

— etc 

2. Cteneral nMnagemenI plan 

— See list of diseases to be recognized 

— Physical rehabilitation for patients who have suffered myocardial 

Infarction, stroke 
— Schedule of Immunization for Infants, special risic groups 
— To minimize health hazards according to age, sex, race, socio- 
economic status and place of residence 

— etc. 

3. Diagnostic plan * synnilomo and signs 

— Fever 
—Pallor 
— Weakness 
— Jaundice 
— Vomiting 

— etc. 



« Extracted from the curriculum of the Abraham Lincoln School of Medtctne, University 
of tlllnolt College of Medicine. 
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alike can be freed from the artificial c<n)straints of instructioiial time 
and sequence. 



CURRICULUM FLANS 

Traditionally a curriculum; n takes tbe form of a descriptive outline 
of courses, their arrangement and sequence, the time as^gned to them, 
the content that will be covered in fhem, and the instructional methods 
that win be employed Such a document guides faculQr in preparing 
thdr instruction, tells students where to go and what e]q)eriences they 
can expect, and provides for external reviewers the kind of evidence 
that is generally souglit when judgements are made about educational 
quality. But It tells none of these individuals or groups ^at the 
product of the curriculum is intended to be. 

The ''accountable curriculum" plan, on the other hand, would also 
be an outline but would focus upon a description of learmng outcomes 
and the varied resources to which students might turn to achieve those 
goals. These resources might indude formal course offerings, where 
they were appropriate and useful, but only as one of many modes that 
nu^t be employed This document would also serve facdty, but as a 

constant remin<fer that there is no -singje-cujEric^m"^)ti m al- fo r ■ all 

students and as regular encouragement to create options and alternatives 
rather than pursue the unreachable goal of the curriculum. It would 
serve students as a definitive guide to what they are expected to 
become, not merely a timetable of what they are expected to do. And 
it would serve external reviewers (as well as internal faculty and 
administration) as a point of reference for judgmg educational quality 
in terms of planned learning outcome rather than designed instructional 
content 

WiOk such a curriculum plan in hand the task of evaluating student 
achievement is substantially simplified, since the first and most critical 
step in that process — specification of learning objectives — will have 
been completed as part of the planning. And this is where evaluation 
must begin, rather than being tacked on at the end. 

But equally important is the opportunity such a plan provides for 
examming the outcome in terms of the social, economic, and political 
as wdl as the academic determinants. In any cateful programme review 
each such determinant in turn should be used a criterion against 
which the curriculum is judged. To the extent that it fails to meet these 
criteria, both the plan and the planning process must be re-examined 
to determine what went wrong. Were the determinants appropriately 
identified? Were the necessary interests and expertise represented in 
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the planning group? Were priorities clearly established and did the 
planners adhere to these priorities in making curriculum deci^ons? 
Were those priority deci^ons reflected in the final curriculum plan as 
well as in the instructional tactics? It is through such systematic review 
that faults are identified and corrected 

But even if the criteria seem to have been met, the task of review 
does not end; instead the cycle should begin once more. Academic, 
social, econonuc, and political determinants are not static; the world 
changes, and with it the curriculum must also change to meet new 
needs as well as to exploit new opportunities. Curriculum theory 
demands that the process of planning be continuous, dynamic, and 
never-ending if it is to serve best the needs of individual students, 
educational institutions, and the sodety to which both are ultimately 
accountable. The time is at hand for that theory to be more fully 
reflected in contemporary curriculum planning practices* 
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SIMULATION IN INSTRUCTION 
AND EVALUATION IN MEDICINE 



Christine R McGuire^ & Frederick H. Wezeman* 



' Sound clinical judgement requires not only a broad kno^edge and 
understanding of health and disease and relevant professional habits and 
attitudes concerning the delivery of health care but also the application 
of complex cognitive and psychomotor sidlls. Hiese skills indude the 
collection^ processing, and interpretation of data; using a variety of 
resources (including expert advice); ordering priorities and decision- 
making; manipulating a situation to alter it (including interacting with 
others to effect change); and monitoring the consequences of these 
manipulations and modifying decisions or actions to respond to the 
altered situation* Conventional methods of instructing and evaluating 
students, residents, and clinicians are often not optimally suited to the 
development and assessment of these important components of phy^ian 
competence. Analogous problems in other fields have sometimes been 
resolved by the introduction of sophisticated simulators designed for 
both teaching and testing. 

For example, witii the advent of the space age, scientists throughout 
the world were made dramatically aware of the need to develop 
simulators capable of creatmg, in tiie gravitational field of the earth, 
tiie conditions and problems that astronauts would encounter in leaving 
that field, living from hours to weeks in a very confined area under 
conditions of zero gravity and finally re-entering tiie once familiar 
environment The resulting simulators represent engineering feats tiiat in 
some respects rival tiie space ships themselves. Yet no one questions 
tiiat tiiese coatiy technological marveln were indispensable: first, in 
assisting scieutists to anticipate and work out alternative solutions to 
problems in a deliberate, step-by-step foshion, free of any pressures that 
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time and danger ordinarily impose; and second, in providing space-fiight 
crews with ample opportunity to practise these solutions in a n^ic-free 
environment, so that all could be assured of appropriate responses in 
the potentially dangerous dtuation of the real universe. 

But space Emulators for training astronauts are merely the most 
recent and techmcally the most sophisticated of a growing number of 
devices that have long enjoyed a deservedly favoured su^.ls as instruc- 
tional adjuncts in the education of such diverse personnd h5 athletes, 
business executives, lawyers, and many other groups. However, despite 
widespread femiliarity with the prindples that underlie these mtriguiag 
approaches, and almost universal aclcnoi^edgement of thdr demon- 
strated efiEcctiveness, simulation tec!mology has only recently been 
introduced in medical education. Perhaps such limited exploitation to 
date is in part owing to the vision of astronomical costs conjured up by 
the word ''^ulatof*. However, recent developments surest tha^ 
within reasonaUe financial limits, this exciting technology can be made 
available to medical educators to enhance the efficiency and effectiveness 
of learning and the validity and reliability of assessment 



Reduced to its essence, simtilation consists merely in placing an 
individual in a setting that imitates some aspect of reality, and deigning 
a problem around that setting which requires that learner's active 
participation in mitiating and carrying through a sequence of inquiries, 
decisions, and actions. The total atuation must be arranged so that 
each of the learner's activities triggers appropriate feedback that he can 
utilize in subsequent decisions about pendhig action — decisions that 
may in turn modify the problem. Certainly one of the most important 
characteristics of simtilation is that a problem can be initiated in 
standard form for everyone but evolve in quite different ways depending 
on the unique configuration of interventions that each respondent makes. 

In continuing experimental work (Rg. 1) this essence has been 
captured in a variety of simulation modalities whose use is economically 
and technolo^cally feasible in the devdopment and assessment of 
critics! components of physician competence. 



The various types of »mtilation outlined below are based on a 
recognition that clinical and laboratory data come to the phyncian 
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The following groups are currently engaged In experimental work 


with one or more typee of simulations for Instruction or evaluation. 
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Michigan State University 


McMaster University 


School of Human Medicine, 
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East Lansing, Mich., USA 


Hamilton, Ontario, Canada 


The Royal College of Family . 


The Australian Academy of 


Physicians of Canada, 


General Practice, 


Toronto, Ontario, Canada 


Melbourne, Australia 


The American Board of 




Orthopaedic Surgery, 




Chicago, III., USA 




At WHO headquarters, Geneva, Switzerland, a demonstration 


lat)oratory of instructional aids Including simulations has been estab* 


llshed under the direction of Dr. M. A« C. Dowling and certain Inter- 


country studies initiated on the relative oHIcacy of some of the simulation 


models described In this paper 





through various sensory modalities (e.g.» visual^ auditory^ tactile) and 
that highly complex cognitive and psychomotor skills are involved in 
collecting and processing such data. These important facts are com- 
monly ignored in much of the teaching and most of the testing 
characteristic of medical education, ^ere all too frequently almost 
exclusive reliance is placed on verbal descriptions of highly complex 
and varied phenomena and processes* 

Reproductions of visual and auditory stimuli 

The utility of photographic reproductions of X-rays» lesions^ gross 
and microscopic specimens^ tracings of electrocardiograms^ or electro- 
encephalograms and the like — all presented in a form that imitates life 
as closely as possible — has long been recognized. More recently 
electronic techniques have been developed for the playback of high- 
fidelity recordings of heart, breathy and abdominal sounds using 
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individual stethophones in a manner that simulates what the physician 
hears during his examination of a patient Similarly, high-quality 
cin6-films or ^deotapes depicting patient interviews and exanunations 
are generally recognized as invaluable for the presentation of data 
consisting of a combination of sound, colour, and movement 

For purposes of instruction, presentation of data in these forms 
makes it possible to contrd the difficulty of the material and to enable 
a student to focus on the essential elements, freed from the multiplicity 
of distractions that real life intrudes. For purposes of assessment, data 
presented in this manner can be employed in standardized test exercises 
that require the student to demonstrate that he has observed the salient 
findings and can describe them accurately, interpret their significance, 
and antidpate other findmgs to which they may be related While such 
methods are being increasingly used in the instruction of the medical 
student, their utility in the evaluation of the student's competence is 
only now beginning to be recognized and is, as yet, inadequately 
exploited. 

Three-dimensional models 

More recently, prototypes of several three-dimensional simulators 
(e.g., breast, pelvis, prostate, eye, ear, larynx, infusion arm, intuba^on, 
spinal tap, etc.) have become available.^ Varied pathology can be 
inserted in these devices to assist the student in learning and to 
require him to demonstrate that he has mastered the skiU of using 
sophisticated instruments to elidt the findings and to intctpttt thent 

For example, the Iowa Eye Model consists of a hoUow styrofoam 
head with realistic, transparent cornea and pupils. The interior of the 
head is fitted with slots into which ordinary 3S-mm slides of the fundus 
are inserted in front of a silvered surface desig!!^! to reflect the light 
from an ophthalmoscope back through the 3S*mm transparency in a 
manner that enables the physidan to visualize the retina, much as he 
would in a real patient Since standard 3S-mm slides are used in a 
manner that makes them readily interchangeable, it is obvious that the 
findings in this model are capable of almost infinite variation. This 
important feature means that for purposes of instruction the findings 
can be carefully graduated in difficulty, while for purposes of assess- 
ment they can be selected to require the student or physidan to 
demonstrate that he can make int^retations ranging from the gross 
distinctions expected of the beginner to the most subtle expected of the 
specialist 



» Pentt. F. A Kolman, S. (1973) /. mid. Edue., 4i, 442. 



Automated robots 



As an indication of the level of realism that can even now be 
attained in three-dimensional models, SIM I (developed at the University 
of Southern California for use by anaesthetists) and the new Cardiac 
Simulator (developed at the University of Miami) may be instructive. 
SIM P is a computer*manage4 extraordinarily life-like robot that can 
be programmed to present combinations of findings capable of modifica- 
tion in an ahnost infinite number of ways, to simidate a variety of 
problems, and to respond appropriately to different interventions of the 
anaesthetist SIM I may even ''die'' if the student manages it 
inappropriately or responds inadequately to changes in its condition. 
The Cardiac Simulator,* although not computer managed, is equipped 
with a large number of programm^^^ presenting varied auscultatory 
findings coordinated with appropriate visual pulses built into ^ life-like 
torso. 

WMIe these robots are at present available only in prototype form, 
they suggest some of the possibilities for using sophisticated simulation 
technology in the trainmg and testing of medical students and 
residents. 



Paper-and-pefwil simutations 

At the other extreme of the cost continuum are the written 
simulations of clinical problems,^ widely known as branched problems 
in patient management (PMP), designed to imitate the decision-making 
prociss involved in the '*work-up*' and management of a patient In 
these problems a clinical situation is introduced with a brief statement 
of a complaint such as a physician might elicit from the patient himself, 
from a friend, firom a relative who. accompanied him, or from a 
colleague who referred him. It may consist of a few written lines or a 
brief length of a film in which Uie patient indicates his reason for 
"seeing the doctor^ 

The statement of the problem is followed by a list of interventions 
or of general strate^es; from these an initial approach may be selected, 
e.g., further history, physical examination, laboratory evaluation, or 
immediate therapy. This decision is recorded by erasing the opaque 
overlay or developmg the latent image on a specially treated answer- 
sheet to reveal the requested information or further instructions. 



s Abrahaouoo* S. et at (196^ /. med. Edue^ U, 515. 

* OordoQ, M. (1971) CompuMud mtomaud maimquln^ a tt^-ustUig method (unpubUatied. 
mllable from ^ UnhcrtitT of Florida School of Medidae. MiamL FUl, USA). 

» MoOtiiie, C A Solomofw L*, ed. (1971) CIMcd simidMions: mUcUd pmhkmi In ptdUni 
mmuignmnt^ New York* ^kum-Centiiiy-CroCtf. 
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In each new section possible interventions that may yldd further 
information about the patient or modify the patients condition are 
listed Hie respondent selects as numy or as few as seem appropriate 
in light of the specific circumstances obtaining at that stage of the 
proUenu As each decision is recorded the results of that intervention 
are revealed in realistic verbal or visual form. As the problem unfolds, 
the student must make further choices based on the specific responses 
of the paient evoked by his own earlier decisions. The compUications 
that the must manage wQl depend, as they do in the consulting roojo^ or 
clinic, on the unique combination of specific procedures that yfrere 
selected at earlier stages. . ' 

This type of paper-and-pencil simulation has been found suitable f ojr 
both diagnostic and management problems rangmg from emergeDey^ to 
long-term chronic disease conditions that may involve psycbbso^ii ai 
well as organic factors. Recent technical developments make it possibly 
for the local faculty to devise such simulations and to reproduce sma^ 
quantities of them in latent image by means available in any secretarial 
office and at a cost comparable with other ine3q>entive duidicatiO|g 
procedures.^ \ , 

ComputerHttded simulations ,^ 

A rationale analogous to that used in the construction of written 
simulations has been employed in the development of a variety of ^pes 
of computer-assisted simulations of the dinical encounter. Althou^* the 
specific details vary, in essence all are designed to imitate some'a$pe6t 
of the decision-making process in the investigation and management 
of patients. 

In some such simulations the student makes selections from a list'df 
options presented by the computer; in others he must fcmaulate his 
own inquiries and interventions with neither the assistance nor the 
constraints of a list For example, m the CASE format, developed at 
the University of Illinois,^ the computer takes the role of a patient 'and. 
the session is introduced by a computer statement describing the'settiUg 
(phy^cian^s consulting room, hospital, emergency room, etc.) and die 
patient's age, sex, and general iq[)pearance. The computer may go on to 
state, as a patient would, the reason why the patient has come to seethe 
physician. Or alternatively the session mi^t sim[dy begin as fdloi^s: 

Student: What brings you to see me? 

^ UcOttif«,~C. ce aL (1974) WritUn tbmdmioHt: thdir eonttmctkm tmd rnidfOi (aapnfeUihcd, 
avtilable Irom Center lor EducatSoiul Development, University ol Illinois Collete of Medicine. 
ChlctiO* ITfrt USA). 

* HftffeM, W. et aL (1971) /. m$4. Zitie^ 443. 
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Computer: WeU, I am just tired all the time* I don't ever feel rested 
and all my energy is gone. 

Student: How long has this been going on? 

. Computer: About »x months ago I started feeling tired all the time. 
I finally felt I had to do something about it 

Student: What did you do? 

In this fashion* the student may question the computer-patient about 
the history and presenting comfdaints and the computer responds as 
the patient would* At any point in this interaction a physical examina* 
tioa or laboratory evaluation may be initiated* In response to each 
sf^ediic inquiry t^ computer provides u technical description of the 
phy^cal findings or the laboratory report requested* At tte condwuon 
of this data collection the student records both his diagnosis and his 
plan of management At this point the computer assumes the role of 
a consultant and immediately responds with a display comparing the 
data actually gathered with the optimal data base and the proposed 
diagnosis and management plan with the consultant's conclusions and 
recommendations* 

In an alternative type of computer programme for simulating the 
patient—physician encounter, developed at the National Institutes of 
Health,^ the student n ust also manage simulated time and cost in a 
realistic manner* Here, the task is to arrive at the correct diagnosis 
with a minimum appropriate expenditure of time and money. At any 
point in the session the diagnosis may be entered; the computer will 
then respond with an evaluation of its accuracy and a summary of 
the time and money cost incurred compared ^th the cost of the 
optimum approach. 

Irrespective of their specific focus and methodology, all computer- 
assisted simulations are designed to provide the stuctent with 
instantaneous feedback on the results of his dedsions and an 
immediately retrievable record of his entire decision-making sequence 
for objective analysis and evaluation. 

Uve^ulations 

Realistic clinical problems can also be posed through live simulations 
hi which an actor, housewife, another student, or almost anyone can be 
"^programmed'' to simulate a patient in an interview setting* In 
principle, live simulations differ from written and computer problems in 
only two respects: first, the student/physician must necessarily generate 

^ Frkdmao^luL 0973) /. mid. Eiue., 4$, 92. 
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his own ioqairies tnd intervtntioiis; and second, he must fonnuUte 
diem in a manner diat is meaninfful to the "^dent^ 

In a simulated diagnosdc interview,^ example, die student maybe 
directed to interview die pcoframmed padent a^ 
in a management or dierapeudc inters it may be die students task 
to persuade the simulated padent to accept a recommended plan of 
manafement In eidier case, die simulated padent is programmed to 
di^y die reluctance, amdety, and need for undersowfinf and 
emotional support diat such padenti di^lay in real life. And it is die 
students task to communicate dearly and convindngly ^di die padent 
in order to gain hb fdl cooperation. 

Skills in data collection, long-term care, or crisis management have 
been explored by means of a variety of such tediniques. In a 
triephone simnkdon, for exan^le, die student is seated in a setting 
umilar to a consulting room when the telephone rings and a frantic 
voice 00 die oUier end <tf die line reports some emergency-^an accident, 
a child suspected of ingesting poison, a sidcide direat, or the like« The 
physidan is required to respond as he would to a rc^4ife emergency 
and the adequacy of Us response can be rdiable analysed, dther for 
purposes of certification or for further leanung. 

live simulations have also been used to teach and to nsseu various 
skills in padent eiumination* Of spedal note in dds regard is die 
experience of Dr Howard Barrows and his associates in programming 
simulated patienU to diq)Iay the physical finings of various neurdc^cal 
disorders*' 

Similarly, die phyridan*s interpersonal slcHs and attitudes can be 
enhanced as wdl as assayed by careftdly derigued sunulated interviews 
and conferences in which an individual— again, not necessarily a 
professional actor — is programmed to take the part of a patient, a 
colleague, or odier member of die healdi team.* Sudi interviews can 
be devdoped to focus primarily on die student's senritiWty and sldU in 
handling various types of patienU; on impc^tant referral, consulUtion, 
and conference skills, including diose required in die delicate dtuations 
in wbkh colleagues differ sharply about the management of a particular 
patient; or on skill in communicating widi odier members of the healdi 
team (eg., instructing a nurse, requesting assistance from dietitian or 
social worker, or malting a presentation in a staff conference). Sudi 
Nidations are uveful in assisting die student to dtwlop, and his 
mentors to evaluate, die range of communication and iaterpersonal 
skills essential to effective functioning as a phyridan. - 
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Iiitliort»rq)roductioi»ctf dii^ diree>dtmeih 
tkNid moddt, automtted rotxMi^ writtea siinulatk)ii$» computer 
rim u U t ioos, and live simulatioos can all be used to present dinical 
problems in a realistic form that (a) enables the student to leam» and 
lequiiei him to demonstrate that he bas learned, how to respond as 
he would in the consulting-room or clinic setting; (fr) pemdts him to 
recdve life-Ulx feedback from the simulaton and (c) forces him 
with the results of a sequence of inquiries and interventioos, much 
as be wm have to do in his actual practice. 

Thott^ simulation technology is still relative^ new to medicine these 
methods are already being emjdoytd in teaching and testing medical 
students, residents, and clinidans. It may in die near future be posulde 
to combine various simuIatiOQ modalities in a manner that fiirther 
enhances their utility. For examine, a ccMnputer ^ulaticm could be 
merged ^th a live simulation so that a constantly updated computerized 
data bank is made availaUe to supfdy information about the pathology 
(e.g., physical and laboratory findh^) that a live simuUtor b incapable 
of providing. Such a simtiUtion couki be extended to cover the ^AiU t^ 
period begini^g with tbe presentation of the patient and cciiduding 
only after appropriate treatment has resolved the problet*; and tbe 
evident changes in the ""patient's'' condition havi; mt suitable criteria 
for discharge. In such a problem changes over time in the patient's 
condition induced by, and resulting from, intervention by the i^ysician 
would be reflected not only in modified findings supjdied by tbe 
computerized data bank but also in changes in ti^ spec^ roles and 
reactions oi the simulated patient and of the physician. Hius, a com|4ete 
clinical encounter could be simulated in a condensed time period But 
merpng live and computer simulati<ms is only one of the many possiUe 
ways of comt»ning all types of rimulation modalities to increase their 
versatiUty and effectiveness in medical settings. 



THE USB OP SIMULATIONS 

In instmctkm 

As indica t ed above, the various simulation methods now available 
in medidne can be employed as instructional resources not only in die 
development of diagnostic and proUem-sdMng skiUs, but also in tbe 
development of psychcMUotor and interpersonal skills. Clearly, these 
emulations can provide more valid educational experience than man/ 
<tf our conventional modes of instruction. However, this increased 
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validity depends upon the fidelity of the filiation, whether it is 
presented in a live» computer, paper-and-pendl, or other format While 
fidelity is a relevant consideration in all emulation methods, the 
proUem is particularly acute in the development of three-dimen^onal 
models, in which it has been espedally ^cidt to amulate tissue 
diaracteristics accurately. \^th the advent of improved materials and 
better production techniques it has now become posable to develop 
sufficiently accurate representations of human rjiatomy for teaching 
some of the sidlls of physical diagnosis. Analogous advances in 
electronics have led to the development of more compact electronic 
simulation devices (such as tiie Humetric -Sound Device) tiiat 
dectrcxiically synthesize heart sounds ami other phenomena in the body. 
These, too, have proved of great value ii3 asdsting medical students to 
develop skill in interpreting certam types o£ phy^cal findings. 

Such innovations afford more efficient means of (a) focusing upon a 
spedfic learning objective; (b) rendering the experience reprodudble; 
and (c) allowing a standardized approach for all students* In the use 
of ^ulation devices it is important to keep dearly in mind the nature 
of the desired behavioural outcomes, ^nce the ultimate goal is the 
subsequent independent performance of the sidll in a non-amulated 
setting. The degree to which a learned skill is transferable from a 
simulated setting to a live patient atuation depends not only on the 
fidelity of (be device itself but also on its incorporation into a systematic 
total educational approach* Mere exposure to a device affords die 
student a chance to experiment with or interact with the device only 
at a superfidal level On fhr other hand, the devetopment of assodated 
programmed instructional /^des that direct the student— device inter- 
action magmfy the relevant aspects of the experience* Thus, statements 
of the goals and objectives of the experience with the device, 
performance skill expectations, metiiodolo^cal instructions, and 
evaluation items expedite the acquisition of the skill, direct the 
interaction with the device, and guarantee a profitable experience. 

In assessment 

The results of current research suggest that simulation techniques of 
the types described here are espedally useful in evaluating (a) skill in 
gatiiering and interpreting clinical and laboratory data; (b) judgement 
in patient management; and (c) the professional skills and attitudes 
required for effective interaction witii patients and colleagues (Table 1)* 

Interpretative skills. Qearty, greater reliability can be placed on any 
judgement about a students interpretative skills if he asked to interpret 
data presented in the form in which he usually encounters them rather 
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TABLE 1. SIMULATION AS AN EVALUATION TECHNIQUE 



Comptt«fic« twt«d 


Type of •Imuletlon 


tltustrative technique 


Intorpratatfve skills 


Simulated clinical 
and latwratory 
data 


Pfiotographic reproductions 
Sound simulators 
Gind-films and videotape 
Three-dimensional models 
Automated rotx)t — SIM 1 


Probfen^sofving 
skills and clinical 
judgement 


Simulated problems 
in patient 
management 


Written simulations (PMP) 
Computer-assisted simulations 

(CASE) 
Automated '^obot simulations 

(SIMI) 

Live simulations (telephone, sur- 
gery, etc) 


Interpersonal skills 
and attitudes 


Simulated interviews 
and conferences 


Programmed patients 
Programmed colleagues 
Programmed teams 



than verbal descriptions of such data. For this reason^ the use of 
photographic reproductions, sound simulators, motion pictures and 
videotape, three-dimensional models, and more sophisticated robots are 
of particular use in assessing this important aspea of clinical 
competence. 

Problem-solving skills and cttnicd judgement Patients oidinarily 
present with a chief complamt, which the phy^dan must explore and 
manage. Thus there is a certain ^&ce validly* to any technique that 
confronts the student with a problem in this form, provides him with an 
opportunity for considerable latitude in choosing his own approach, 
and enables him to receive prompt feedback on the results of any 
inquiry or intervention he makes. Qearly it will be more useful in 
assessing the student's clinical judgment than the more traditional 
exercises that ask him to list the signs and symptoms of various 
diseases, and the most probable diagnosis from a carefully predigested 
summary of the salient data. 

Professional skilk and attitudes. Too often in ths past, medical 
educators have either failed to evaluate psychomotor and interpersonal 
skills or have attempted to make inferences about these essential 
components of competence from the studenf s verbal description of his 
interaction with a patient or from the written record of a patient 
investigation. Clearly, both sources of data are inadequate for a valid 
assessment of skills and attitudes. These can be judged only by observing 
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the student with a patient However, since the student may behave 
differently with different patients and since patients must not be put at 
risk from incompetent interventions, observation of the student with 
simulators, both mechanical and live, would seem to be the most 
promising alternative. 



CRTTERU FOR USING SIMULATION 

In instruction 

Certain criteria for the use of simulation devices have been implied 
in the preceding (Uscussion; it would be possible to identify others that 
are especially pertinent to the instructional uses of simulators. To 
catalogue them, however, would be unrewarding since many relate 
merely to common sense and good judgement Rrst among these is the 
requirement that the development and use of a simulator parallel 
curricular needs. This assurance, together with the advice of members 
of the teaching faculty, guarantees that experience with the device 
will be a supportive extension of the stated curriculum, as well as 
one that gathers support for further application of such aids to the 
teaching programme by the faculty members themselves. 

No matter how clearly the experience coincides with stated curricular 
needs, or how apparenfly acceptable it may be to teaching faculty and 
students, other sources of difficulty may jeopardize the validity of an 
experience with simulators. Such problems may arise from the physical 
nature of the devices themselves. In the area of anatomical replications, 
commercial development in the past has poorly satisfied the need for 
instructional accuracy in terms of texture, size, apparent "tissue" 
density, or mobility. The resulting lack of structural reality sets an 
incomplete stage for the interaction of the student with the device. 

In addition, careful attention must be ^ven to the level of education 
in question, the placement of skill learning in the educational con- 
tmurnn, and the knowledge that must be attained before experience with 
a specific simulation device is of maximum benefit Obviously, the 
simple skill of performing a pelvic examination can be acquired without 
understanding the normal or abnormal pathology and physiology of the 
female reproductive system. Appreciating the importance and value of 
either the simulated or real experience demands a knowledge of basic 
sciences, pathology, and some fundamentals of obstetrics and 
gynaecology* Thus, it is of greater value to expose a student to a 
simulated learning device at an appropriately defined time in his 
training. 
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In assessment 

In selecting simulation techniques for purposes of evaluating 
individual competence and/or curricular effectiveness, the same con- 
siderations apply as in the choice of any other testing and evaluation 
technique. As with the instructional uses of simulation, it is first 
necessary to decide exactly what aspect of competence is to be 
measured at each level of education and experience. Once this decision 
has been made, then the most appropriate technique is that which 
requires the student to perform a task resembling as closely as possible 
the real-life performance for which he is being examined (test validity). 
Secondly, it is necessary to design the task in Guch a manner that 
different observers who are evaluating the student will agree about the 
character of his performance in the test situation (objectivity or inter- 
rater reliability). Third, it is essential to create a sufficient number and 
variety of tasks to ensure that one can safely generalize from the 
student's performance on the test exercises to his overall level of 
competence (sampling reliability). Finally, both economic feasibility 
and logistic considerations must be taken into account in selecting 
any assessment technique. These four criteria should be borne in mind 
in reviewing the advantages and limitations of the simulation tech- 
niques elaborated upon below. 

ADVANTAGES AND UMfrATIGNS OF SIMULATION 

Simulation imitates^ it does not duplicate y life; this is both its greatest 
advantage and, in the view of some, its greatest limitation. Qearly 
there are some aspects of reality that cannot at the present time be 
economically simulated, if they can be simulated at all. Thus, it is 
necessary to be very cautious about predicting on the basis of responses 
to a simulated situation how an individual will behave in reality; at best 
it is possible to predict how well that individual is capable of behaving. 
(The same caution must, of course, be observed in predicting how an 
mdividual will behave in a new life situation on the basis of how he 
behaved in a prior, necessarily different^ life situation). 

Furthei, it is important to recognize that simulation is not an 
appropriate method for teaching or for measuring all aspects of 
performance. For example, factual information is more economically 
conveyed and more directly measured by conventional techniques of 
instruction and testing. On the other hand, appropriate professional 
habits are most firmly entrenched by repeated reinforcement over a long 
time-span in diverse settings; they are most reliable assayed by careful 
and repeated observation under similar conditions. Between these two 
extremes simulation provides the following important advantages. 
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Perceived relevance. As compared with more conventional teaching 
and testing techniques, simulation offers the possibility of designing 
instructional and assessment exercises so that they correspond more 
closely to the situations a physician actually faces. Such materials 
appear to the student as far more relevant than typical readings, 
lectures, multiple-choice tests, or traditional oral quizzes. Such perceived 
relevance is at the very least of psycholo^cal and motivational benefit 

Predetermination and preselection of the task. It is important to note 
that this relevance can be achieved without dependence on the flow of 
patients available at the particular place and the specific moment in time 
that instruction is to take place or assessment is to be made. Thus 
simulation makes it possible to predetemune precisely the exact task 
that students are to learn or to demonstrate that they can perform. 
Further, by suppressing the undesired distractions always present in 
reality, it is possible in a simulation to focus on the elements of primary 
concern and to eliminate irrelevant, confusing * complexities that 
complicate learning and contaminate assessment 

liie fact that simulated tasks can be preselected also means that they 
can be carefully graduated in difficulty as a student or resident 
progresses tbrou^ training — a lector of prime significance in 
increasing the efficiency and effectiveness of both instruction and 
evaluation. Finally, by developing standard parallel simulations it is 
possible to confront a student again and again with challenging varia- 
tions of what is essentially the same task until he has mastered it and 
can demonstrate that mastery to any examinmg body. 

Standardization of the task. Just as a single student can be repeatedly 
confronted with the same task, simulation enables an examining body 
to standardize the task for all examinees, and to do so without 
subjecting one or a few patients to repeated harassment by a large 
number of students. In short, all examinees can be ^ven exactly the 
same problem to cope with and this can be accomplished without risk 
to any patient 

Improved sampling of performance. By standardizing the tasks and 
focusing on the most significant aspects in each it is possible in a ^ven 
time period to expose the student to, and to sample his performance on, 
a much broader and more representative group of problems. This, of 
course, requires careful selection and definition of the tasks, but that is 
precisely the luxury simulation affords that reality can rarely provide in 
a reasonable time frame* 

Improved rating of performance. When the exact tasks that are to 
compose an instructional unit or an evaluation exercise are precisely 
defined and preselected it is possible to record the students 
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performance in detail for subsequent feedback in a manner that 
maximizes his learning. Further, for purposes of evaluation it is possible 
to develop specific criteria for jud^g performance, and to train 
examiners in applying these criteria equitably and consistently. Thus the 
degree of inter-rater reliability that can be achieved in scoring examinee 
performance with simulated problems and simulated interviews is far 
superior to that achievable with real patients. 

Increased responsibility and realistic feedback in a practical time 
frame. One of the most important advantages of simulation over reality 
consists in the fact that all students and residents can be allowed full 
responsibility for the work-up and management of the sunulated patient 
without any risk whatsoever to actual patients. Even if the student's 
interventions with the simulated patient are grossly inappropriate he 
can be allowed to continue without supervision, to receive realistic 
feedback, and to learn the consequences of his action from first-hand 
experience, without harm to anyone. The simulated patient can develop 
complications induced by faulty management, get very much worse, die, 
commit suicide, and yet be repeatedly revived, tirelessly to serve in the 
edification and examination of the same or other students. 

Compression of real time. Furthermore, it is often essential to provide 
a student experience with, and to test his mastery in the handling of, 
chronic disease without his spending months or years observing the 
gradual deterioration of a real patient. In carefully developed simulations 
a lifetime of chronic disease can be compressed into u half-hour 
problem, at each stage of which the student can be provided with 
feedback about his interventions in a form that is more instructive 
than life itself usually yields. 

Increased learmng. The nature of the feedback provided in simula- 
tions as compared with that available in either conventional educational 
settings or in reality should be stressed. While the former rarely provides 
any genuinely relevant feedback, the latter often furnishes it only 
indirectly, equivocally, and after a long delay. In contrast, the prompt, 
specific, and unambigious feedback characteristic of Wf^U-designed 
simulations mates them a powerful tool for the enhancement of 
learning. 

RELATIONSHIP OF SD^ULATION 
TO OTHER INSTRUCTIONAL AND EVALUATION STRATEGIES 

Enumeration of the foregoing advantages should not be construed as 
a recommendation that conventional, instructional and ev^^uation 
.strategies be entirely replaced by simulation techniques; nor should it 
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be interpreted as suggesting that simulation is a pale, but adequate, 
substitute for the real thing when the latter is unavailable. Rather» 
simulations should be viewed as important adjuncts to other educa- 
tional methods. Tlieir power lies in the fact that they select out 
manageable segments of the total physician/patient relationship for 
study. However, th>s ^'^ry fragmentation, in turn, requires that provision 
be made for the student to experience through conventional patient 
contact the total interaction at appropriate points in his professional 
development 



CONCLUSION 

In summary, the evidence from studies of the use of simulation at 
several levels of medical education and with a variety of types of 
groups suggests that when these techniques are properly exploited* 
simulation technology gives considerable promise of increasing the 
effectiveness of mstruction and the validity and reliability of assessments 
of more relevant and complex objectives of medical education. Simula- 
tion appears to offer special advantages as an effective technique not 
only in teaching and assaying specific, circumscribed skills but also in 
developing and evaluating that most significant objective of all — 
clinical judgement 
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EXAMINATIONS AND DECISION-MAKING 
James A. Bobula^ 



One of the most challenging tasks facing medical educators is that of 
developmg examinations that measure accurately and assess appropri- 
ately the adequacy of student knowledge^ skills^ and professional 
attitudes. This evaluative task is of such importance because it is chiefly 
through examination data that teachers monitor student progress 
through meJical school, identify those who are to qualify as physicians, 
and thus ultimately control both tlie quantity and quality of the 
physicians available to society. This paper deals in an introductory, 
non^-iechnical fashion with the systematic planning, scoring, and 
reporting of examinations to serve those critically important functions. 

No single examination can provide adequate data for all the decisions 
a faculty must make about medical students. Each examination, how- 
ever, is most likely to serve a particular evaluative need if it is designed 
firom an explicit statement of the decision a faculty must make and is 
developed to provide the specific data required for that decision. For 
example- 

A Proficiency Examination will be administered to incoming first-year students 
to determine whether any student has sufficient mastery of: (a) one or more 
disciplines within the basic science curriculum to warrant either placement into 
advanced courses or exemption from regular courses in such disciplines; or 
ib) the basic medical sciences as a whole to warrant placement directly into a 
clinical curriculum. 

Given such a statement of purpose, the particular examination 
characteristics that are consistent with the decision to be made must be 
specified and the examination planned on the basis of those specifica- 
tions. The example summarized in Table 1 is further developed below. 



^ Center for Educational Development, University of Illinois College of Medicine, ClUcago, 
L, USA. 



3d 



35 



TABLE 1. EXAMINATION PUN 



ExamlntUon chmcttrittics 



Proficltnc}' Maniination exampit 



1. Scope of material covered 



The content will systematically represent the 
entire basic science curriculum, according to 
a blueprint developed by a special faculty 
committee composed of one representative 
from each basic science discipline. 



2. Format Multiple-choice questions with only one 

correct answer will be employed exclusively 
In the examination. 

3. Nature of data A student will receive a score Indicating the 
(i.e., type of scores) percentage of his answers that were correct 

In each discipline and In the examination as 
a whole. 



A minimum passing percentage score will be 
established by the faculty committee prior to 
the examination for each discipline and for the 
total examination. A student will be considered 
at least minimally competent in a particular 
discipline or In the basic sciences as a whole 
tf he obtains a percentage score equal to or 
greater than Its corresponding minimum 
passing percentage score. 

5. Critical review The faculty's Appraisal Committee will review 

the examination blueprint prior to adminlstra* 
tion of the examination to approve its 
specifications. After the examination Is ad* 
ministered, the Committee will assess the 
reliability (Internal consistency) of the exami- 
nation and will perform an item analysis to 
determine whether any of the multiple-choice 
questions were ambiguous to the students and 
must be deleted from scoring. 



4. Basis for decision 
(i.e., standard of 
adequate performance) 



6. Reports A report of each student's percentage scores 

and the minimum passing percentage score for 
each discipline and for the total examination 
will be made available to the student himself, 
to his faculty adviser, and to the faculty com* 
mlttee responsible for deciding on advance 
placements. 
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THE DEVELOPMENT PROCESS 



Step 1: state tlte purpose 

A statement of purpose should indicate the reason for which an 
examination is being constructed and administered, specifying (a) the 
group of students to be examined, (b) the areas on which they are to be 
examined, and (c) the decision that the staff must take on the bads of 
the examination results. 

Specification of the group of students to be examined enables 
definition in broad terms of the skills reqmred (e.g., students at the end 
of the first year and of the fourth year mi^t both be expected to know 
something about cardiac arrhythmias, but at quite different leveb of 
competence). Specification of die content about which the students are 
to be examined tells what particular areas are to be dealt with in this 
examination. Specification of the decision to be made about the 
students mdicates how the examination data are going to be \ised in 
evaluating student performance, i.e., to monitor a student^s progress 
(diagnostic examinations) or to judge competence at some dgnificant 
transitional point (certif^g examinations). 

Diagnostic examinations assess mastery of a specific unit of instruc- 
tion so that a decision may be made whether or not further learning 
experiences are necessary. Their purpose is to advise students in 
planning subsequent learning activities. I^agnostic examinations include 
pre-tests to determine what a student knows about certain topics before 
they are formally treated in the curriculum, mid-unit tests to assess 
progress during instruction, and post-tests to determine whether the 
expected levels of proficiency have been achieved by the end of an 
instructional unit 

Certifying examinations, on the other band, assess mastery of an 
entire area of study so that a decidon may be made on promotion, 
graduation, or licensure. Certifying procedures include placement 
examinations^ wMch permit a student to bypass formal instruction in 
areas that he has ahready mastered; comprehensive examinations, which 
document a student's achievement of profesdonal competence at the 
end of a formal course of study; and licensmg examinations, which 
testify to the ri^t of professional practice. 

Step 2: determine what is to be examined 

In order to achieve a specified purpose, it is necessary to develop a 
detailed blueprint oudining both the content of the examination and the 
performance processes the students are expected to demonstrate with 
respect to that content ''Content'' refers to the body of knowledge, set 
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of skills, or focus of attitudes on which the students are to be examined. 
"Performance processes" refer to what the students are expected to be 
able to do with the content being examined In the area of cognitive 
performance processes, for example, students might be expected: (1) to 
simply recall information ("What is X?"); (2) to interpret data usmg 
this information ("Knowing what X is, what do these data mean?"); or 
(3) to solve a problem by fitting this information together with other 
information in reaching a conclusion about a complex phenomenon 
("Knowing the nature of X and its relationships to A, B, and C, what is 
the nature of Z?"). 

An exanunation blueprmt is best presented as a table of specifications 
that lists relevant content areas on one axis, relevant performance 
processes on the other, and the proportion of the examination to be 
devoted to each content/process combination within the cells (Table 2). 
The proportions generally reflect the relative importance of these 
elements: more important areas are given greater weight in the examina* 
tion; less important areas receive a smaller portion or are eliminated 
entirely* 

TABLE 2. EXAMINATION BLUEPRINT : CCGNnVE DOMAIN 



Performance processes 



Recall Interpretation Probiem«solvlno 

Content areas Vo of 30 40 30 

examination 



Area-I 


20 


12 


8 


0 


Area 2 


35 


5 


10 


20 


Area 3 


10 


3 


4 


3 


Area 4 


20 


8 


8 


6 


Area 5 


15 


2 


12 


1 



The development of a blueprint and preparation of test materials 
according to that design mcrease the probability that «he examination's 
content will be consistent with its purpose and that the performance 
data will provide a sound base on which to make the decisions for 
which the examination is intended. 

Step 3: determine the format 

"Format" refers to the types of exammation materials and methods 
selected to measure the specified contents and processes. Formats are 
broadly categorized as '*free-response** or "fixed-response-" types. Free- 
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response methods, which allow a student to respond to a question or a 
situation with his cwn words or actions, include oral examinations, 
extended^response written questions (essays), and short-response written 
questions (sunple direct questions, sentence completions). The 
unstructured description of student behaviour by faculty observers is 
also a free-response type, although in this case the response is made 
by the observer and not by the student himself. Fixed-response 
materials, which require adherence to a predetermined form, include 
true/false, multiple-choice, and matching questions, patient management 
problems, and checklists or rating scales completed by examiners 
observing student behaWour in either an oral examiuadon or a natural 
performance setting. 

Free-response examinations have the advantage of requirijig a 
student to respond independently, drawing solely on his own resources^ 
They have the disadvantage of being virtually impossible to correct and 
score mechanically and are often subject to such extraneous influences 
as an examinee's general verbal fluency, an examiner's lack o£ clear 
criteria for essential elements of a correct response, or differences in 
judgements made by various exammers. Fixed-response examinations, 
on the other hand, can be scored mechanically or by any examiner with 
a listmg of the correct responses. Their major disadvantages are that 
students might respond correctly by guessing or might be cued to correct 
responses by the options offered Since skilful development of fixed- 
response formats allows testing of many complex as well as simple 
objectives of medical education, the additional advantage of objectivity 
and ease of scoring recommend them highly for most examination 
purposes. 

Selection of a particular format should be determined primarily by 
the degree to which it requires a student to perform the desired process 
in the specified content area. For example, when the purpose of an 
examination is to assess the student's competence in obtaming a 
patient's medical history it is more appropriate to have a trained 
examiner with a fixed-response checklist observe him takmg a history 
than to ask him, ''What questions should be used routinely in obtaining 
a patient's medical history?" 

Fuially, an examination need not be limited to a single format The 
blueprint can prescribe the use of different formats within an examina- 
tion wherever they are most appropriate. 

Siep 4: develop a scoring system 

The purpose of a scoring system is to summarize performance in 
some symbolic form that can serve as a basis for rational decision about 




the adequacy of that perfonnafice* Selection of the spedfic method 
should tegin with a dear statement of exactly what the score is intended 
to mean* Such a statement mi^t read: ''A studenfs score in each 
disdi^ine induded in the Proficiency Examination will indicate the 
percentage of the total content of that disdj^e that he has mastered** 

In oHer for a score to have tins meaning, the e>:amination blueprint 
must ensure that the total content of each dlsdpline is systematically 
represented Unless it is the score cannot indicate a mastery of the total 
disdpline but only the components that were induded 

Assuming that the blueprint supports the scoring purpose^ it will then 
be necessary to assign some numerical value to each response a student 
makes* Since the form of the response depends upon examiaation 
format, assignment of values must be consistent with that format If, for 
examine, multiide-choice items having a sm^e correct response are 
emi^oyed, then a correct response mi^t be ^ven a value of 1 and 
any other a value of 0. Multiple-choice items hamg several conect 
options would require a different procedure. For example, with three 
correct responses each might be assigned a value of 1 (so that a student 
could receive values of 0, 1, 2 or 3 for the item) or a value of 1/3 
(so tiiat a student could receive values of 0, 1/3, 2/3 or 1 for the item). 

Finally, the values assigned to each response must be capable of 
summarization in a single score by means of some mathematical 
formula Uiat clearly expresses the intended meaning. In the Profidency 
Examination example, a formula meaning ^'percentage of the total 
content of a discipline that a student has mastered*^ would be: the sum 
of the values assigned to each of the student's correct responses in a 
discipline divided by the sunl oi the vducs assigned to all correct 
responses in that discipline. Following this formula, if a student 
correctiy answered 96 of 120 multiple-choice items with single correct 
responses in a given disdpline, his score would be 96/120, or 80%. 
The meaning of this score is: "The student has mastered 80% of the 
content of this disdpline.'' 

So long as the desired meaning of a score can be stated and that 
meaning supported bv the examination content, the method of assigning 
values to correct responses, and the summary formula, it is possible to 
generate scores assessing nearly any skill expected of the students. 

Step 5: set standards appropriate for the decisions to be made 

Since examinations are the principal method of determing the 
adequacy of student performance, each aspect of performance about 
whidi a decision is to be made must be assigned an appropriate score 
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in accordance wlm step 4 of the examination plan. These scores provide 
the data from which judgements will be made. In step a standard of 
acceptable performance is established for each component of the 
examination in order to provide faculty with a basis for a decision 
about whether individual performance is adequate or inadequate. 

There are two principal sq>proaches to the establishment of 
performance standards: normative and criterion. In the former, an 
individual student's performance is assessed ^niv in comparison with 
the performance of all others taking the examination. In the latter, 
the adequacy of an mul^idual student's performance is determined in 
relation to an absolute standard established before the examination 
is admmistered. 

Whatever the method, application of a standard to a student's score 
results in the assignment of an assessment symbol. For example, based 
on the total scores in a comprehensive examination the faculty might 
wish to award special recognition to the three top students. Here a 
normative standard is employed, ranking performance according to total 
score and giving special acknowledgement to the first three ranks. In 
another case, the faculty might merely wish to determine who **passes'* 
and who "foils''. Here a criterion standard is employed, and each 
student's score k cempared ?ath the minimum acceptable performance- 
level, resulting in an assessment symbol of P (pass) or F (fail). 

Table 3 illustrates several sets of commonly used assessment symbols 
that might be assigned. to the performance of twenty medical students 
in a single examination. Each set of symbols is further explained 
below. 

Normative standards 

1. Rank: the place an individual score occupies in an ordered listing 
of all scores sequenced according to magnitude. The symbol 1 indicates 
first, 2 indicates second, etc. If more than one person has a particular 
score, all persons with that score are assigned the same rank symbol, 
e.g., S.5 is assigned to students e and /, who together occupy the fifth 
and sixth rank positions. 

2. Percentile rank: the percentage of students with scores below a 
particular score. The symbol 80 for student d indicates that 80% of 
the students had scores below his. 

3. S^scores: the number of standard deviation units by which a 
particular score departs from the group average (mean) score* The 
standard deviation is a statistic that describes the variability in a set 
of scores. The z-score of 1.08 indicates that student c is 1.08 standard 
deviations above the group average (the standard deviation in this 
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TABLE 3. ILLUSTRATIVE ASSESSMENT SYMBOLS 











NotmaUve 








Student 


Score 






























Rank 


Percentile Z-score 


Letter 


Pess/Fall 


Letter 


a 


89 


1 


95 


1.50 


A 


P 


0 


b 


87 


2 


90 


1^ 


A 


P 


0 


c 


85 


3 


85 


1.08 


B 


P 


0 


d 


81 


4 


80 


.67 


B 


P 


s 


e 


80 


5.5 


70 


.57 


B 


P 


s 


f 


80 


5.5 


70 


.57 


B 


P 


s 


9 


79 


7^ 


60 


.46 


C 


P 


s 


h 


79 


7^ 


60 


.46 


C 


P 


s 


1 


77 


9 


55 


.26 


C 


P 


s 


i 


75 


10.5 


45 


.05 


C 


P 


s 


k 


75 


10.5 


45 


.05 


C 


P 


s 


1 


74 


12 


40 


—.05 


C 


P 


s 


m 


72 


13 


03 


rye 


C 


P 


s 


n 


71 


14.5 


25 


—.36 


C 


P 


s 


s 




14.5 


-25 


- - —.36- 




- P 


. s 


K 


69 


IRS 


15 


—.57 


D 


p 


s 


q 


69 


16.5 


15 


—.57 


D 


P 


s 


r 


68 


18 


10 


—67 


D 


F 


u 


s 


65 


19 


5 


—.98 


F 


F 


u 


t 


44 


20 


0 


—3.15 


F 


F 


u 



illustration is 9.68, the mean is 74.5, and the student's score is 85). 
Z-scores are useful in comparing a student's relative position in the 
group on a series of different examinations. 

4. Letter-grades: in this case the symbols A, B, C, D, and F indicate 
excellent, very good, average, poor, and unsatisfactory respectively, and 
are based upon percentile ranks so that the categories include 
approwmately the following distribution: 10% A, 20% B, 40% C, 
20% D, and 10% R 

Criterion standards 

1. Pass/ fail: the assessment symbol P is assigned to all scores at or 
above the predetermined passing score, and F to those below. For this 
examination, the faculty determined that a score of 69 would be 
required for passing. 

2. Letter-grades: in this case, the faculty agreed in advance that 
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scores of 84 and above would be considered outstanding (O), between 
69 and 83 satisfactory (S), and 68 or below unsatisfactory (U). 

The standard selected for judging student performance must be 
consistent with the purposes of the examination. In a Proficiency 
Examination designed "to determine whether any student has 
sufScient mastery of one or more disciplines within the basic science 
curriculum or of the basic medical sciences as a whole/^ it would be 
inappropriate to apply a normative standard. It is entirely possible that 
none of the students is proficient enough in all disciplines to merit 
certification. The only sensible standard would be a predetermined 
criterion requirement — a minimum passing score established in 
advance as the minimum acceptable evidence of mastery of the 
examination's content If the standard employed in evaluating student 
scores is not consistent with the purpose of the examination, it is 
impossible for a faculty to make sound decisions. 

Step 6: establish procedures for examination review 

Even the best efforts of skilled examiners do not always yield a 
perfect examination, one that accurately characterizes the abilities of 
the students. Consequently it is necessary to establish procedures to 
evaluate the examination itself before reportmg results. Important areas 
for such evaluation include the following. 

Adequacy of content. The examination blueprint should be reviewed 
to determine whether the examination covers die intended content and 
process categories in a manner consistent with the purposes of the 
examination. The examination materials should also be reviewed to 
determine whether or not they meet the specifications set forth in the 
blueprint These analyses are best undert^en before the examination 
is administered and by persons sufficiendy familiar with the curriculum 
to make judgements on content adequacy (but not t^^e same persons 
who developed the blueprint and examination materials). Whenever 
deficiences are discovered, the content must be improved before the 
examination is given. 

Clarity of items. It is also important to review individual items for 
conceptual and verbal clarity. This is accomplished not only by 
pre-examination editorial review but also by statistical item analysis 
after the examination is administered. 

For example, the author of a multiple-choice question may hold 
the following opinion of the S possible responses: 
option 1 — inconect, easily recognized as such; 
option 2 — correct; 

option 3 — mcorrect, easily recognized as such; 



option 4 — incorrect, diflScuIt to distinguish from correct; and 
option 5 — ^incorrect, somewhat difiScuIt to distinguish from correct. 
From this description it would be expected that very few students 
would select options 1 and 3» that a larger proportion would select 
option 5y and that even more would select options 2 (the correct option) 
and 4; and actual results might than take die form shown in line A of 
Table 4. If, however, the results showjx in line B were obtained, the 
author might suspect either that the students do not know the content 
or that there is some deficiency — some ambiguity — in the item itself. 
Any items found to be ambiguous by this post-examination review 
should foe deleted from the examination before final scores and assess* 
ment symbols are assigned. 

TABLE 4. ITEM ANALYSIS 



Option 1 2 3 4 5 



A. Vo students 2 58 4 25 11 

B. students 2 31 4 25 38 



Consistency of results. Two general methods of assessing reliability 
(consistency) may be employed: repeated ^.dm'nistration of the examina- 
tion to see whether it assigns similar scores to each student each time 
(test— retest methods); and comparison of parts mthia the examination 
to see whether scores assigned to a student by one part are similar to 
scores assigned by other parts (internal consistency methods). For 
example, an examination with low reliability as determined by internal 
consistency methods might have several items that many of the better 
students (with high total scores) missed and the poorer students (with 
low total scores) answered correctly. Such an examinatioa would be 
improved by deletion of these items before final scores are computed 
for the students. 

Because it is important that both faculty and students can have 
confidence in the decisions made on the basis of an examination, it 
is essential that every examination be adequately evaluated. Any 
examination that fails to meet technical standaords of adequacy cannot 
serve as a rational basis for decisions about student performance. 

Step 7: report the results 

The final step is that of reporting the results to all persons involved in 
the decisions being made on the basis of that examination. Exactly 
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what is reported and to vrhom depends upon the purposes for which 
the examination was administered. 

For example^ the results of a diagnostic examination aimed at 
assessing a student's mastery of five content areas and his ability to 
perform three cognitive processes would be sent to the student and to 
his faculty adviser, showing a separate score and/or assessment symbol 
for each content area and for each cognitive process. This set of scores 
would then enable the student and his adviser to pinpoint specific 
weaknesses that warrant special attention. Because of the diagnostic 
nature of the examination, no report would be entered on the student's 
permanent record. 

In the case of a certifying examination it may only be necessary to 
report the student's total score and its associated assess: lent symbol, 
since here the purpose is usually to record overall competence and not 
the specific components of competence. This information would be 
reported to the student and to the faculty committee responsible for 
certification and become part of the student's permanent record 

In any case, the contents of the reports and persons who receive 
them must be consistent with the purposes of the examination, to ensure 
that all relevant information is reported appropriately to all (and only 
those) persons entided to knowledge of it. - • - • - 
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DYNAMICS OF A LEARNING GROUP 



Shirley Perlmutter* 



The knowledge explosion of the last 25 years has vastly increased 
the subject-matter to be taught in today's medical schools, and the 
demand for trained physicians has also accelerated. Medical schools 
are often understaffed and overextended, yet faculties must face students 
who have become increasingly vocal in demanding that classroom 
content be more relevant to the skills needed on the job. The medical 
schcfbl teacher &f tCxIay fiftds himself facing "a dilemma ver}''much like " * 
that of the sorcerer's apprentice, whose every effort to stem the tide 
merely sent the waters higher. The purpose of this paper is to propose 
one way by which health professions teachers may be released from the 
demand that seems to be imposed by the imending growth of knowledge 
and heightened student expectations. To attain this goal, some of the 
assumptions underlying traditional teacher practices must first be 
examined in order to explore alternative approaches. 



TRADmONAL TEACHER BEHAVIOUR 

Traditionally, the behaviour of teachers in medical schools has been 
characterized by a high degree of control over and direction of students, 
a posture presumably based on the assumptions that (1) if left to their 
own devices students would be apathetic about or resistant to the task 
of learning; (2) students do not know what they need in order to 
achieve effective professional performance goals; and (3) the teacher 
has the necessary expertise to determine the nature and scope of the 
subject-matter to be mastered. Thus it is commonly regarded as a 
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facility responsibility to motivate and direct students, to control their 
actions and influence their behaviour in such ways that the educational 
goals will be achieved. Teachers see themselves, and are seen by their 
students, as the experts who will present material and determine the 
extent to which students have mastered it With the pressure of time 
and a constantly expanding factual content, teachers are rarely 
encouraged., or provided with the opportunity, to slow down and 
examine more closely the implications of these teaching practices for 
learning. 

While this traditional approach to teaching is undoubtedly an 
efficient means of transmitting large quantities of information to large 
numbers of students, there is a real question of whether they grasp 
what is presented. It is also clear that such an instructional format 
produces students who are dependent upon experts for direction rather 
than independent in the pursuit of learning that must be lifelong. They 
come to believe that all expertise does reside in their teachers, and that 
performance should be judged by the expert standards that teachers 
alone establish. It should come as no surprise, then, that students are 
particularly concerned with how their behaviour is perceived by 
teachers, and express urgent needs to be directed, assigned specific 
"tasks, 'and told that they are carrying out tfiose 'tasks in the proper 
way. When deprived of these accustomed external sources of support 
and direction it is also understandable that they may experience 
considerable anxiety, if not frank hostility. And all this occurs in an 
atmosphere that is regularly described as impersonal and lacking in the 
important element of close interaction sought by both students and 
teachers. 



A NEW FORMAT 

As it becomes increasingly apparent that students can no longer be 
taught all they will need to learn, the focus of education must shift 
to teaching students how to learn and how to be independent, 
self-directed, and equipped to deal with an ever-chan^ng environment 
It has been said that if a student can be trained to be an effective and 
independent learner he need not be filled with all the information he 
can contain before going into a new job. He will have the capacity to 
learn whatever he needs to learn. Indeed, he will have to generate that 
capacity in any case, whether he is trained to do so or not, since no 
classroom can prepare him for every contingency, for every aspect of 
life and work encountered outside the classroom. 

Training students to become self-directed learners implies a set of 



ERLC 



47 

47 



postulates about students and the learning process different from those 
described earlier. It assumes: (1) that each student enters the learning 
situation with an inherent desire to learn and a capacity to accept 
responsibility for his own learning; (2) that he has tmtapped 
resources that he may utiUze to identify his own learning needs, 
assess potential learning resources available in his environment, and 
participate mfh teachers in determining the content to be mastered; 
(3) that significant and rapid learning will occur when he perceives the 
subject*matter as relevant to his purposes; and (4) that encouraguig 
students to confront in an educational setting the kinds of practical 
problems they must face in the real world is one of the most 
effective modes of promoting learning. 

One place in which to foster such independent learning is the small 
group. The idea of reducing class size to improve the quality of 
learning is not new. It has often been proposed in the hope that 
smaller numbers will allow more opportunity for student— teacher 
interaction, for the quiet and shy student to participate more freely, and 
for all students to become more actively involved with the content. 
But the expectation that a reduction in class size ynH bring about these 
changes is unrealistic. A one-way flow of communication from teacher 
to students can occur in a small group as well as a large one*— and 
often does. If the potential of the small group (10 — 20 students) is to be 
realized fully, then traditional teaching behaviour must be modified to 
match the assumptions about students and learning listed above. In 
order to accomplish successfully this different teaching stjde, a teacher 
must be knowledgeable about the dimensions of the group process, the 
developmental stages of groups, and how to condv himself in a 
manner that supports group development and the achievement of small 
group goals. 

DIMENSIONS OF THE GROUP PROCESS 

In order to avoid any possible misunderstanding it should be noted 
at once that group discu5sion is not intended to bf; random but 
rather to deal with some identified content. But ^tever the content, 
all groups develop an operational process — a style of working together 
that has great impact on the quality of the product — and a set of 
relationships — patterns of interaction among the participants that affect 
the character of the experience. Whether the teacher recognizes them or 
not, these factors are at work, either facilitating or impeding the 
achievement of group goals. It is the teacher's task to be sure that the 
former, rather than the latter, is the dominant mode. If he is sensitive to 
this process he will give particular attention to (1) the amount of 
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individual participation; (2) how members try to influence each other; 
(3) how group decisions are made; (4) how subgroupings are formed; 
and (5) how feelings are expressed. 

In all groups, regardless of the purpose for which they have been 
formed or the nature of their participants, the process evolves through 
six general stages: 

(1) Stage 1, getting acquainted, is characterized by polite and 
superficial interaction between the members. Even when they have 
met before, or even worked together, the formation of a new group 
creates an unfamiliar setting in which members must get to know 
each other anew* 

(2) During stage 2, What am I here for?, members experience a 
period of ambiguity, a lack of direction or purpose. They become 
frustrated, angry, confused, and impatient 

(3) Out of this sense of disorder stage 3, a bid for power and 
influence, emerges. Members be^ to attempt to create order out of 
what has become intolerable confusion. It is at this time that working 
rules for the group begin to be established and commitment to 
membership begins. 

(4> In stage 4, What can 1 4o Jo help?, group cohesion appears. . 
Natural resources and leaders are identified and members negotiate 
around who will do what, and when to achieve the group task. 

(5) Stage 5 is characterized by a sense of accomplishment. Through 
the negotiations that occuned during the stage 4 members are able to 
collaborate, to execute and complete the task. 

If a group is formed for the purpose of accomplishing more than one 
task, each time a task is completed and another proposed the group will 
recycle through the first five stages. While the stage of getting acquainted 
may be less obvious the second time roimd, even this does occur. A new 
task usually creates the need for different resources and behaviours that 
were not reqmred by the previous t£sk. Thus as new tasks arise 
members must get to know each other in new ways. 

(6) Fmally stage 6, termination, occurs when the members recognize 
that at a specific time the group will come to an end and the participants 
will disperse. During this last stage (such as the final week of a 4*week 
workshop or the last month of a school year) members begin to dis- 
engage from the group and to reduce their mvolvement with and 
commitment to group issues. Some time may be spent ui this stage 
reviewmg events during the life of the group, in an attempt to 
summarize and put a closure to the experience. 

One of the functions served by the model just outlmed is that of 
providmg a perspective for certain cyclic phenomena inherent ui group 
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work. For example, a let-down feeling and sense of disorganization 
often follow the "high" feelings associated with the sense of 
accomplishment in stage 5. This reappearance of "disorganization*' is 
frequently interpreted as indicating poor group functioning, when it is 
in fact a natural phenomenon that recurs during the life of any group. 
The model also underlines the sequential nature of the stages and the 
importance of each. Even the period of ambiguity and chaos in stage 2 
is productive. If members can tolerate some degree of frustration and 
uncertainty and refrain from prematurely organizing the group (stage 3), 
opportunities for potentially innovative and creative problem-solving 
around tasks will be enhanced. 



CONDUCTING AN EFFECTIVE GROUP 

In such small groups the ideal teacher is one who more often listens 
than talks, questions rather than answers, moderates rather than 
dictates^ and stimulates others rather than performs himself. In such a 
setting the personal commitment to learning becomes overt and 
shared; both^ students and teacher dan ask for feedback; personalized 
two-way communication can develop among students, and between 
students and teacher; and evaluation can be immediate, ongoing, and 
multidimensional from self, peers, and teacher. 
- In order to help readers to gain some concrete notion of how these 
principles can be translated into practice, a sequence of events is 
described below, illustrating how a teacher might facilitate the 
development of a climate that encourages self-directed and independent 
learning in the small group setting: 

(1) The teacher opens with a description of his goals by presenting an 
overview of the objectives, the content to be mastered, and a proposed 
schedule of activities. Additional staff resources may also be introduced 
or otherwise identified. 

(2) After this general introduction students are asked to describe 
their goals. These expectations are explored together through a problem 
census depicting individual learning needs. 

• (3) Teacher and students then compare the general programme 
objectives with the individual learning needs, noting similarities as well 
as differences. 

(4) Learning needs identified by bo:h teachers and students are 
translated into specific objectives aud lis^^d in order of priority. 

(5) Teacher and students then identify jointly the objectives they can 
reasonably expect to achieve during the programme. 
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It is impoitwit that these five activities occur at the ouisct, in ordtx to 
establish as a norm the active dialogue^ among sudents themselvf;s as 
well as between students and faculty* The five steps assume that while 
the faculty has a set of general objectives and an instructionai strategy 
to implement their achievement, there is sufficient flexibility in the 
programme to allow responsiveness to student input This dc^ not 
imply that students determine what will be taught but that the oppor- 
tunity exists for shifting emphasis from one topic to another, for the 
inclusion of a related issue not originally planned, or for the elimination 
of something already learned, for example. It is also worth noting that 
the five steps correspond to the five stages of group development The 
staff introduction and statement of goals vis-d-vis the student expres- 
sions of individual learning needs and expectations is a way for feculty 
and students to begin to get acquainted (I). The expression of individual 
learning interests and needs is often elaborate as well as diffuse, so 
that students and teachers alike may experience a ssnse of confusion 
(II) and of being overwhehned (How will all this get done?). Com- 
paring student and teacher goals, listing them according to perceived 
importance, is a way of creating order (ID) out of the confusion. 
Negotiating (IV) a jomt statement of^oals also involves nercssaiy 
collaboration (Y). 

This sequence, which generally takes 2—3 hours, illustrates the basic 
steps required to buUd a cooperative learning group. It may seem 
excessive in a class that meets for only one hour three times a week 
for a term, but the time invested in this way at the begpming wiU pay 
handsome dividends in the rapidity and solidity of subsequent learning. 

But not all small groups enjoy die luxury of an extended period 
during which participants may work together; some may meet only once, 
and then for a short period. Yet even in the course of a single 
hour a skilful teacher may put into practice some of the principles that 
have been cited as critical contributions to productive group work and 
to the long-range goal of encouraging self-sustained independent 
learning. An illustration of such an effoit, one that was created and 
tested by a participant in a recent teacher-training programme, is set 
forth in the accompanying table. 

A tcacher*s primary concern, when he is trying to adapt small-group 
learning methodology to settings where the content is relatively fixed, 
is to devise a means of actively engaging students in the learning 
process. Even when time and content do not permit the group to evolve 
through five stages, the model may be usefil in identifying ways in 
which the learning activities may be designed tc elicit maximum student 
participation. 

For example, in the biodegradation exercise (see the accompanying 
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aiODEQRADAT10N*-SMAa*QROUP EXERCISE* 



ObMhm: At the conclusion of the 1-hour session students will be 
able to: 

(1) Identify the reasons for modem sewage disposal; 

(2) predict the effect on the ecology of a large lake of the 
release of a large volume of untreated or partially 
treated sewage; 

(3) predict the possible effect on the health of a com- 
munity of the release of a large volume of untreated or 
partially treated sewage In nearby water courses; 

(4) evaluate data on the biochemical oxygen demand of a 
specified sample of water. 

Detlgn: (1) Introduction. 

Teacher Introduces the topic and provides an overview 
10 min of the hour. 

(2) Class breaks Into groups of 3. 

Handouts distributed on Lake Erie ("Death of a l^ke**) 
and on gastrointestinal outbreaks. 
15 m!n • Task -Pi*t the diseases spread vie fiew»ge . 

- understand the concept of biochemical oxygen 
demand. 

(3) Return to full group — one member of each trio sum- 
marizes the deliberations under 2 above. 

20 mIn (4) Short lecture, illustrated with visual aids, followed by a 
discussion of sewage treatment. 

(5) Class forms two groups of 6. 

Handouts distributed on biochemical oxygen demand. 
10 mIn Task -to understi id tests and interpret their signifi- 
cance. 

(6) Return to full group. 

Final summary and distribution of a synopsis of the 
session. 



• Reproduced by permission of Or Adrian Lee. Department of Medical Microbiology, 
Faculty of Madlcln#. Unlvtralty of New South Wales. Sydney. Australia. 

figure) the teacher began with stage two. When he had identified himself 
as the authority with the intention of directing students to a specific 
topic, stage 2 was not an issue for the full group or the smaller 
groupings. The real group work was initiated with stage 3, when the 
class was divided into trios, each of which was required to designate 
a reporter. Stages 1 and 2 then occurred briefly as members began to 
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know each other through the ways in which each responded to the task 
in hand, and as they sought to ascertain whether there was any residual 
confusion about the task. But the empha^ in stage 3 was on identifying 
resources within the group, so that they could progress through stages 
4 and S to completion of the task. This sequence would then be 
repeated when groups of six were formed for the second task* Stage 6, 
termination, occurred when the groups reassembled as a full class for 
the final summary and distribution of a synopsis. But at every stage the 
focus was upon the students and their responsibility for learning, not 
upon the teacher and his opportunity to instruct. 



CONCLUSION 

Self-direction and independent learning are important general 
objectives in any programme of medical education, but this does not 
imply that even the most vigorous proponents of this stance advocate 
anarchy, in which every student does what he wants to do, any more 
than it is tolerable for every faculty member to do only what he wants 
to do without regard for others. There is a place for lectures and drill, 
for studeiy, listening as iveU as faculty talking. But if individu^d learning 
is to be optimized, faculty members must be prepared to exploit the 
principles outlined here, whether they are working with a few or with 
many students. It is, of course, more (tiflScult with large numbers, but the 
teacher who accepts the fundamental validity of the assumptions upon 
which such an approach is based will find ways to incorporate those 
principles wherever he teaches. Subgrouping a large class to allow 
smaller numbers to engage even briefly in an active process of learning 
may seem a waste of precious time — but only if the teacher's role is 
seen as information source rather than learning facilitator. To the extent 
that they encourage students to find in their peers some of the stimulus 
and guidance that in conventional instruction depend upon teachers 
alone, they create an ever-widening group that can continue to pro\dde 
mutual support in the act of learning independently and outside the 
classroom. And that, of course, is where it must occur for the rest of a 
professional career. 
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TEACHING LARGE GROUPS 

George R Miller^ 



As higher education througihout the devdoped world has evolved 
from the privileged experience of a small elite to the reasonable 
expectation of a substantial segment of contemporary youth, a growing 
volume of complaints are heard about decline in quality of that 
experience as the number of students increases. Today, in developed 
and developing countries alike, those responsible for health professions 
education programmes are confronted with the problem that the social 
demand for trained professionals has dictated a substantial increase in 
student numbers in the absence of a proportionate expansion in what 
are believed to rq)resent minimal resources of personnel and facilities^ 
There is no faculty dissent from the demonstrable fact that larg^ 
numbers of students can be processed throu^ these educational 
programmes; their concern arises from the ^parent impediment large 
numbers pose to producing graduates who have acquired not only the 
necessary body of information about health and disease but also the 
capacity to apply that information successfully in the management of 
the medical problems they must ultimately face, and the attitudes of 
personal responsibility for continuing their own self-education if the 
socially unacceptable alternative of professional obsolescence is to 
be avoided. 

The issue is how to resolve the educational problem created by a 
discrepancy between large numbers of students and limited resoiurces 
with which to train them. Rational analysis suggests three possible 
solutions: 

1 . Reducing the number of students is probably not a viable approach. 
It would be difficult to mount any convincmg argument that large 



^ Director, Center for Edacfttlcnal Development, Unlvertlty of nUnols CoUtn of Medidse, 
Chicago, lU, USA. 

55 

Er|c 53 



numbm decrease educadonai quality^ for the limited evidence available 
suggests (hat class size is not the critical variable in the quality of 
individual leaming.^*^ In addition there is still to^ mudi to be 
accomplished in improving man's health, too much faith that this cannot 
be done without a substantial increase in the number of practitioners, 
and too much prestige assodattd with the i^ysician role, to expect diat 
sockty will reverse a trend that has won such widespread support 

2. Increasing the resources seems an equally unproductive path to 
pursue. This does not imply that medical sdiools hava all the resources 
tiiey need or that faculties should relax their efforts to acquire more. 
But the economic fact is inescapable that medical education is already 
among the most expendve of university programmes and there are 
otiier important claims on that portion of the gross national product 
of any nation that can legitimately be assigned to education or to health. 
There will certainly be continuing discussion of the issue of funding 
but it i& unrealistic to expect an eariy solution to the problem of 
numbers as a result of a substantial and widespread increase in 
resources. 

3. Vdng the available resources differentiy. Since the major resource 
in ai^ educational enterprise is the foculty, whose members are surely 
capable of learning how to use themselves differentiy, this approach 
would appear to have the greatest immediate and long-term potential 
utUity. The areas of teaching staff change that nug^t be most fruitfully 
explored are: (a) altered perception of the teaching role; {b) the employ- 
ment of different instructional strategies and tactics; and (c) the 
development of new instructional skills. 



THE TEACHER ROLE 



Medical teachers have generally wou teaching appointments only 
after long study and the achievement of recognition as biomedical 
scientists or medical practitioners. Under these circumstances it is 
understandable that so many teachers should see their principal function 
as tiiat of sharing with students tiie knowledge tiiey hav^ acquired, 
the skills they have developed, and the wisdom they have won. The 
teacher role thus becomes one of telling and shomng, and the student 
role that of listening and watchmg until the teacher's learning has 
been transmitted. 



^ us Govenuneot Printins Office (1960) Effectlvmss In ieachlnt. New dimensions In higher 
education. No. 2* Washington. D.C. 
* Stnazaro. P. J. (1966) Oass size in medical school. /. med. Educ., 41. 1017. 
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Such a role may be appropriate in a static field where final (or at least 
semi-fioal) truth has been established and the student task is simply one 
of absorbing fixed patterns of behaviour. It is surely not appropriate 
in a rapidly developing field where "facts" are always chan^g, new 
professional skills are constantly emer^g, and the student's task is that 
of learning how to weig^ and judge both the new and the old throughout 
his professional career. Here the teacher's responsibility is not to 
transfer his knowledge to students but to join them in a never-ending 
search for knowledge. In some instances what he has learned may prove 
useful to them» in others their challenges may prove useful to him^ but 
in either mstance the exchange is in both directions^ not merely one way 
from authority to apprentice. 

Many teachers who are intellectually prepared to accept such a role 
find it difficult to translate mto practice. If they are not to engage 
primarily in showing and tellings what are they to do? Three major 
tasks can be clearly identified: 

1. Helping students to define the objective of a learning experience. 
Anyone who has tried to "assist" a neophyte to learn anything knows 
how frustrating the work can be for both parties. The teacher who loses 
patience or fails to understand the purpose of the effort may respond 
too quickly when students cry» "just tell me what to learn and I will 
learn it" Only as a teacher comes to accept as his primary role that of 
helping students to learn how to learn, rather than to learn what he 
knows» will the interaction become both fruitful and satisfying. Once a 
student clearly understands the educational objectives in terms of his 
own career goals, his learning will generally proceed at a pace that 
astonishes both himself and his instructor. 

"People learn what they want to learn" is an old aphorism too often 
overlooked by teachers who forget that their most important function 
is that of helpmg students to discover a need to know (i.e., the 
objective) — and then getting out of their way. 

2. Helping students to identify the available learning resources. 
Among Uiese resources are certainly individual teachers, but if there 
is honest acceptance of the principle that all knowledge does not reside 
in one teacher — or one department — theii students will be helped to 
discover and use the many others that can be found on all sides. The 
most generally available resources are the printed materials collected 
in libraries^ now increasingly supplemented by a growing array of other 
learning aids such as audiotapes, slides, fihns, videotapes, and 
programmed devices. 

The problem is really not one of availability (although in some parts 
of the world the supply of books and journals may be limited) but of 
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acceptance. Teachers who see their major teachmg function as that of 
transmitting information often find it difficult to identify a source as 
accurate^ as currenti or as complete as their own knowledge^ and thus 
are unable or unwilling even to recommend a textbook — imless they 
have written it More consistent with the role highlighted here is the 
attitude of teachers who recognize the ever-changing nature of knowl* 
edge and are comfortable with having students learn things they may 
not personally accept — even things they may think wrong — provided 
that at the same time those students also learn how to make judgements 
and to defend with logic and reason the conclusions they have drawn. 
Such teachers will help students to see» appreciate^ and use the infinitely 
varied learning resources a medical school provides, not merely the 
faculty member who stands before them. 

3. Providing students with non-judgementcU feedback on the extent 
to which they are achieving the learning objectives that have been 
agreed upon* The keyword in this phrase is "non-judgemental." To be 
sure, it is essential for quality control of education to make a judge- 
ment at some point of the extent of the individual student^s learning, 
but the best evidence now available suggests that any student y/ho 
has the intellectual ability reqmred for admission to medical school 
is able to master its programme of study if given appropriate guidance 
and adequate time. The most important guidance function a teacher 
can fulfil is that of regular and systematic determination with each 
student of progress towards educational goals — a joint appraisal of 
what has been learned and i^at has yet to be learned, so that sub- 
sequent study can be directed towards filling the remaining gaps 
rather than further refining what has already been achieved. If such 
periodic, non-judgemental feedback is accomplished effectively, a 
student can face the ultimate judgemental assessment confident diat 
he is abe to meet the minimum performance requirements. 

Assumption of the teaching role described here and employment of 
the instructional strategies enumerated may demand a fundamental 
change iu the attitudes of many teachers. Not least, there must be a 
willingness to accept students as responsible individuals who are able 
to learn independently and who need support more than instruction. 



NEW INSTRUCTIONAL TACTICS 

It is almost inevitable that, when encoimtered for the first time, the 
views set forth above will be interpreted as an attack on the lecture 



method of instructioiL The removal of emphasis from "teaching" (in the 
sense of telling) and the placing of emphasis upon individual and 
independent learning certainly suggest such a conclusion. But it is 
important to draw c sharp distinction between the lecture method of 
instruction, and the lecture system of education* It is the latter^ not the 
former, that must be discarded if the broad educational goals set 
forth in the opening paragraphs are to be achieved. A teacher who 
abandons the role of primary information source and accepts the role 
ot learning facilitator may still use lectures but will use them sparingly 
and in a manner that serves gpals more general than the transfer of 
information* 

At the same time it must be acknowledged that in some parts of the 
world printed or audiovisual instractional materials in the native tongue 
of most students are virtually nonexistent Here the alternatives seem 
to be: (a) the use of lectures or (b) the provision of printed information 
in a second language. Since books are often beyond the means of 
individual students the lecture system may thus by default become the 
dominant instructional mode, but such a system is attended by very 
serious hazards. Quite apart from the dependent learning habits that it 
encourages, anyone who has read carefully the literature on lectures will 
be appalled by the inaccuracies that occur as the lecturer's notes become 
the student's notes and by the serious limitation on infoilnation 
absorption that occurs after a relatively short tune (15—30 minutes). 
Under these circumstances it is probably wiser for teachers to record 
their words for transcription in the native tongue or for playback by 
individual students or groups. This will at least introduce some measure 
of flexibility into the programme and an opportunity to use for other 
instmctional purposes in later years the time that might be invested 
in repeating the same lectures. Despite the feeling teachers often have 
that lectures must be updated each year, the fundamental concepts to 
which such instmction for medical students should be addressed rarely 
change so rapidly. 

But whether lectures are used as the principal method of instruction 
or merely as one element of a more varied instructional programme, 
the method can probably be used far more effectively than it is in most 
classrooms today. A change in dominant lecture style is especially 
important when the student body is large and opportunities for 
individual encounters with teachers are limited. A skilful lecturer can 
convey even to an audience of 500 some sense of personal interaction 
if he will avoid the impulse to tell all he knows, interrupt his discourse 
with concrete and relevant illustrations, and challenge his audience 
periodically to apply what has gone before or anticipate what is to 
come. The details of how to modify and improve lectures have been 
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described elsewhere^; the purpose here is merely to remind readers that 
even this most widely used instructional method can be employed to 
serve more general objectives that are particularly significant if the 
problems associated with large classes are to be met successfully. 

There are other methodological tactics that require no additional 
resources, merely a willingness to accept the new role ^'Iready outlined 
A simple illustration is the distribution of a clear and concise statement 
of learning objectives for a programme or course before instruction 
begins. Such a statement must be more meaningful than, for example, 
"to appreciate the importance of tranquillizers in the treatment of 
emotional disturbances". It must be couched in terms of what a student 
must do to ensure that both he and his teachers know that he has 
gained such appreciation. The methods of deriving and stating educa- 
tional objectives have also been dealt with elsewhere^; it is the use to 
which they are put as instructional tools that is of concern here. Too 
often students are placed in the uncomfortable and frustrating position 
of having to guess the objective of an educational offering and waste 
untold energy in trying to anticipate what a teacher really wants them 
to learn (not to mention for what purpose). When the objective is both 
specified and shared in advance many students find ways to accomplish 
that learning in a manner more closely matched to a personal learning 
style than the faculty-designed method, and often in significantly less 
time. In the case of large groups, making such provision for individual 
differences is particularly important 

If it is also possible to provide a wide array of aids to independent 
study, so much the better. First among these are printed materials, 
available in such rich variety in so many parts of the world. More 
sophisticated instructional technology such as single concept films, 
audio*cassettes, programmed instructional devices, and simulations may 
have greater appeal but what is important is not the device itself but 
how that tool is used. Too often these aids to learning are regarded as 
add-ons, mere supplements to the conventional source of wisdom, the 
"teacher". If a new sense of personal responsibility for individual 
learning is ever to be achievul by health professions students, these 
instruments must be integral elements of a flexible and individualized 
programme, not merely extras to be enjoyed if time permits. 

Among the most readily available, yet least used, aids to such an 
educational enterprise are the students themselves as teachers. Staff 



^ Bughman, E. (1973) The lecture method of instruction. In: Development of educational 
programmts for the health professions, Geneva, World Health Organiiation (Public Health 
Papers, No. 52) p. 52, 

* MUIer, G.E. (1973) Educational objectives. In: Development of educational programmes 
for the health professions, Geneva, World Health Organization (Public Health Papers, No. 52), 
p, 26. 
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may not be prepared to acknowledge it, but the evidence is persuasive 
that a substantial amount of the most persistent learmng that occurs in 
any educational programme results from peer interaction. Rather than 
allow this to take place in unplanned and random fashion, teachers 
who are willing to accept the primary role of guide might harness this 
vast potential to serve defined educational objectives. Encouraging the 
more advanced students to help the less advanced, the more senior to 
tutor the more junior, not only multiplies the instructional resources of 
an institution but also provides those who teach with an unparalleled 
opportunity to further their own learning. A fifth*year medical student 
may not be able to provide for a second*year student the depth of 
instruction in biochemistry that a biochemist would offer, but the 
communication is more likely to match the learning readiness of the 
novice. It will at the same time crystallize his own understanding of the 
topic, for in the most succesful teaching/learning exchange the teacher 
profits quite as much as the taught Such factors should encourage the 
utilization of students as teachers in any setting; where classes are large, 
the further advantage of reducing the size of learning groups, through 
increasing the number of teachers, is also achieved. 

While peers may provide the most readily accessible pool of 
additional teaching talent there are others, rarely used, from whom 
health professionals can also learn: the practitioners of these professions. 
So long as the teaching role is regarded principally as that of informa* 
tion source it is understandable that academics at the frontiers of a 
discipline are reluctant to entrust students to practitioners who are not 
always **up to date." But if the teacher becomes a guide, one who helps 
a student to identify what he needs to learn and who provides oppor- 
tunities through which that learning can occur, then practitioners can 
become important contributors to this effort What they have to offer 
may also be considerably closer to the career for which most students 
are preparing than the contributions of academic specialists, who now 
play the dominant role in most programmes of medical education. 

Finally, if such non-traditional activities are to further the most 
effective learning and the most eflBcient use of learning resources there 
must be some ongoing mechanism for students and faculty to assure 
themselves that learning is being accomplished. It is here that the 
diagnostic examination fulfils such an important function. Students 
everywhere have been conditioned to perceive examinations as a threat 
to their academic survival, and omfortunately too many teachers still 
use examinations in a manner that justifies such a conclusion. It may 
take time, and will certmnly require that teachers behave more in 
keeping with the role of facilitator rather than judge, but with these 
provisos there is no reason to doubt that students can be brought to 
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the point of seeking out diagnostic examinations to help them define 
more precisely both what they have learned and what they have yet to 
master. The mechanisms for implementing such a programme have 
been dealt with at length elsewhere* and need not be repeated here. 
But it may be important to emphasize that acconq)lishing the aim of 
increasing individualization of learning opportunities for large classes 
is unlikely unless there is some provision for regular, personal, non* 
judgemental feedback of learning achievements and remaining de- 
ficiencies* 

NEW SKILLS 

Even teachers who are persuaded that they must accept a new role 
and are willing to try new instructional methods will probably discover 
that their preparation has not provided the necessary skills for con- 
ducting educational programmes so different from those they have 
known. Iix fact, there is little hope that what has been outlined here can 
be achieved unless teachers are willuig to develop new pedagogic skills 
and further refine some they may already have acquired (e.g., devel- 
oping greater proficiency in delivering lectures). 

Among the most vital is the skill of educational planning. It is 
probably no exaggeration to say that most medical teachers now 
develop an educational plan on the basis of subject-matter content 
Either through personal exptrience with the field in which they are 
expert or throu^ the guidance provided by standard textbooks, they 
decide what must be taught, and in what sequence. But the goal of 
education is learning, not teaching; and the distinction is more than 
semantic: learning implies a change in behaviour, not merely exposure 
to instruction. Thus educationalplanning must begin with clear delin- 
eation of what students should be able to do at the conclusion of a course 
of study, after which the knowledge they need can be more rationally 
defined. Smce the primary purpose of health professions education is to 
produce practitioners who are able to promote the health of society, 
this definition will relate, directly to the health of the conmiunity they 
are to serve. The derivation and specification of objectives, and the 
mechanism for their uicorporation into educational programmes, has 
also been described i^isewhere^* ^; it is to these descriptions that teachers 

xMcGuire, CH. (i973) Dtagnosttc examtnationi in medical tducatfon. i;?: Developnunt of 
educational progro'/nmes for the health professions, Geneva, World Heailh Organization 
(Public Health Taimrs, No. 52), p. 59. 

* Miller, O.B. (1973) Educatlonai obfectives. In- Development of educational programmes 
for the health professions, Geneva, World Health Organization {Public Health Pape.'s, No. 52), 
p. 26. 

* Pochyly, D.P. (1973) Educational programme planning. In: Development of rducational 
programmes fcr the health professions, Geneva, World HealJi Organization (Pulfllc Health 
Papers. No. 57.), p. 17. 
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ready to acquire new skills for dealing with large classes may wish to 
turn for further guidance. 

Obviously, if individualized instruction is to be made available to a 
large body of students it will be necessary to produce or to select many 
new aids to learning. The means of accomplishing these goals have also 
been described.^ It is important to insert a cautionary reminder here. 
The purpose is not merely to develop new materials that exploit the 
mechanicai potential of a growing array of simple and complex devices, 
it is to facilitate learning. Tools have no value in and of themselves. 
If oicjr are misused their potential to impede learning may be realized 
Thus in this arena the skill that teachers must acquire is not only one of 
developing suitable materials to aid learning, but also of using them 
appropriately as an integral element of a total learning system. 

Most important, however, may be the ability to provide individual 
students with the encouragement, guidance, and support they need in 
order to learn most successfully. Although such skills are important in 
any educational environment they become critical elements in a 
programme that must accommodate large numbers of students and 
where the opportunities for individual or small-group exchanges 
between teacher and student are so limited that each encounter must be 
maximally effective. 

The techniques for organizing and moderating small-group discussion 
have already been dealt with.^ Rather than repeat here what has 
been said more fully there, only three items will be mentioned 
that seem to be central to the dialogue which, to facilitate learning, 
must encourage students to disclose their ignorance, assist them to 
discover a need to know new things, and acknowledge the achievements 
that they demonstrate. In order to accomplish this a teacher must 
develop 9 high level of skill in questioning thoughtfully, listening 
perceptively, and responding supportively. 

It may seem strange that the first of these skills is questioning, since 
quizzing is already extensively used in educadonal programmes for the 
health professions. It is important, however, to focus upon the difference 
between interrogation^ which is certamly widespread, and inquiring^ 
which is far less commonly practised. One takes the form of a 
catechism, where question and answer match a predetermined pattern 
and substance; the other takes the form of exploration, in which 
question and answer are the be^nmng of a search, not its end. In the 
former a teacher sets the questions and the student is expected to 



» Penta, F.B. A Telder, T.V. (1973) Audiovisual aids to teaming. In: Devetopment of 
edueatkmal programmes for the healih professions, Geneva, World Heiaitt Organization (JPublie 
HeeUh papers, No. S2), p. 40. 

s See p 46. 
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provide correct responses, while in the latter a teacher may set the stage 
for questions but does not set the limits. The interrogator is interested 
primarily in assessing the knowledge a student has acquired; the 
inquirer is more concerned with identifying how he can be helpful in 
furthering a student's learning. Where the former, following a specific 
reading assignment on perforated peptic ulcer, might ask "What are 
the signs and symptoms of...?*'^ die latter would ask "What still puzzles 
you about...?". The interrogator seeks facts, the inquirer looks for 
reasons. 

The second critical element is the skill of listening. It must be 
surprising to any objective analyst of formal education to discover how 
completely the process is dominated by teachers talking, as though 
listeniag were equivalent to learning. Communication is the keystone 
upon which the educational arch is built, but to be effective the flow 
must be in two directions. This requir^^s teachers not only to be quiet at 
more than infrequent intervals but also, by thoughtful listening and 
inquiring response, to encourage students to question and challenge as 
fully as they are being questioned and challenged. The skilful listener 
is also sensitive to messages commimicated by more than words — to 
the non-verbal expressions that may signal anxiety, hostility, lack of 
interest) or preoccupation vnth other things — and will use these data, 
as well as the words, in fashioning a response. Students may prefer to 
avoid what is not verbalized, but the purpose of the interaction is not 
to do what they want but to do what is necessary to facilitate their 
learning. 

The supportive skill thus comes into focus. Support is provided by 
encouragement and approbation, which should be ^ven sincerely and 
as often as is justified. But support is also provided by revelation and 
critique offered with kindness and without judgement Few students 
want their deficiencies highlighted and it is particularly regrettable that 
few faculty members are skilled in accomplishing this necessary task 
in a manner that is perceived as helpful rather than censorious. To 
provide critique without conveying negative criticism is the goal, for the 
one fosters further leammg while the other merely arouses a defence 
reaction. Until a teacher acquires this skill his efifectiveness in the role 
of educational counsellor and guide will always be limited. 



CONCLUSION 

The problem of large class size, which is encoimtered in so many 
medical schools in both developed and developing countries, is unlikely 
to vanish in the foreseeable future and the associated educational 
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problems are unlikely to be resolved by the massive infusion of new 
resources. The most realistic approach to the question seems to be that 
of using differently the resources now available, and particularly the 
most critical of these — the teacher* 

While this paper has drawn attention to a different rde, alternative 
educational stratepes, and refined pedagogic skills that individual 
teachers can employ to increase their effectiveness in dealing with large 
numbers of students in a more individualized manner, no concrete 
proposal for altering curriculum organization has been made, since 
structure seems less important than the process by which learning is 



FACULTY MAN-HOURS REQUIRED FOR A 20-WEEK BASIC SCIENCE COURSE 
FOR 300 STUDENTS 



Conventional organization Non*traditlonal organization 



Senior staff Atalttanta Senior ataff Aasistants 



Lecture (full class) 
delivery 100 20 

preparation 200 40 



Laboratory^ 

delivery ao 320 50 

preparation 40 160 20 100 

Independent study'' 
(identification and 

preparation of 80 190 

materials) 



Individual conference^ 
(office hours) 100 200 

Small-group workd 
(30 per group) 100 



Total 
man - hours 



420 460 380 540 



900 900 



* laboratory— 4 hours per weeic. 5 laboratory group'i in a conventional programme. Lat>ora- 
tory avaiiable at scheduled hours for students* uie, staffed by assistants only In non- 
tradltlonal organization. 

^Time necessary to identify or prepare texts, articles, or seif-testlrtg devices for 
students to use Independently m home. In the library, in the laboratory, or In the 
classroom. ^ , 

• Provision for each student to have two dO-mlnute Individual conferences during the 
20<^week course. 

^ Provision for one-hour small-group conference every other week for each group off 
30 students. 
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facilitated^ But structure can impede that process and the accom- 
panying table offers an illustration of how a specific leciure/laboratory 
programme for 300 students might be modified to achieve the goal of 
greater individualization without any change in the amount of faculty 
time devoted to the educational tasks. There is ample evidence to 
support the view that such practices are eficctive.^ The question of 
whether teachers are willing to adopt them cannot be answered with 
a comparable degree of confidence. Unless they do, however, it is 
unlikely that the educational problems of teaching large groups will 
ever be solved. 



^ Rodin. A.a A Levioe, H.G. (1973) A med. Educ., 48» 349. 

* Postlethwaite. S.N. (1965) Plant science: a workbook with an audio program approach, 
Minneapolis, Burgess. 
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THE EVALUATION OF TEACHidRS 
AND TEACHING EFFECTIVENESS 



Christine H. McGuire^ 



Fundamentally, the goals of education are not unlike the goals of 
therapy, i.e., to facilitate optimum functioning of the organism. Hence 
the methodology for evaluating teachers and teaching is, in essence, 
analogous to that employed in evaluating any other therapeutic agent, 
and the pitfalls of research in this area parallel those in any field 
of clinical research. 



A DESCRIPTION OF CURRENT APPROACHES 

The three approaches to the evaluation of teachers and teaching that 
are currently most widely employed and that appear to hold the greatest 
promise can be distmguished in terms of the primary object towards 
which assessment efforts are directed: perceptions, pr -'^ess, and product. 
As sliown in Table 1, each approach has its analogue in clinical research 
and, like the latter, its characteristic data type and source. The methods 
of collectmg these data, their interpretation, and their limitations are 
outlined below. In this brief synopsis, discussion is limited to a definition 
of the approach and the identification of the question or questions each 
is designed to answer. 

Assessment of learner perceptions 

Although the oldest, this approach is widely used and very 
commonly misused. It is designed to answer -be question: "How did 
students feel about the quality of the teacher, the teaching, and the 
programme?" Formal or informal student reactions and/or ratmgs 



^ Associate Director, Center for Educational Development, Unlvenity of nUoolt CoUefe of 
Medicine, Chicago, IlL, USA. 
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TABLE 1. APPROACHES TO EVALUATION OF 
TEACHERS AND TEACHING 



Approach 



Data source 



Ciinical equivalent 



Assetsment of Reports of student 

perceptions reactions 



Evaluation of patient's 
subjective feelings 



Assessment of Observations of 

process student*teacher 
interaction 



Evaluation of therapeutic 



rationale on patho- 



physiological principles 



Assessment of product, Tests of student 
outcome, or effect performance 



Evaluation of therapeutic 
efficacy 



constitute the data for answering the question. Typically, tb^^ data are 
collected by interview and/or questionnaire that may range from highly 
structured to completely open*ended inquiries. These may be couched 
in a form that invites the student to comment directly on his reactions, 
or in a form that induces him to respond to a series of mdirect probes 
in a r^anner that reveals reactions about which he would be less able 
or Ic J willing to be candid. 

Assessment of process 

lu contrast to the assessment of perceptions, which deliberately seeks 
opinions about the merits of various elements in the instructional 
programme, process assessment is designed to determine what actually 
occurred in the encounter between teacher and learner. The basic 
question that it attempts to answer is: ''What was the nature and quality 
of the setting and of the communication (interaction) between student 
and teacher?" 

The information needed to answer that question is simple descriptive 
data obtained from systematic, impartial, skilled observation of the 
teacher and the learner in the instructional situation. Since, however, 
there is an unmanageably large number of attributes of the learning 
situation (ranging from tht temperature of the room to the 'lieat" of the 
student-teacher exchange) on which observations could be made, it is 
necessary to select those characteristics likely to be most important in 
facilitating or inhibiting achievement of the desired outcomes. This 
decision is, at least implicitly, based on a theory of learning, and thus 
the observations made are, in fact, designed to answer the question: 
*To what extent does the teacher create an environment in which the 
actual conditions for learning are congruent with those that a selected 
theory of learning asserts as optimal?*' 
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Assessment of product 



Whether a recommended set of conditioiis for learning is, in fact, 
optimal can be determined only by examining their consequences. It is 
to this issue that the assessment of educational product or outcome is 
addressed. All product assessment is designed to answer the question: 
"To what extent is the educadonal agent effective in accomplishing the 
educational goals?" If it is true that teaching is undertaken mainly 
for the purpose of producing student learning, it follows that teacher 
effectiveness must ultimately be assayed in terms of the extent to which 
this intended outcome is achieved, and teacher efficiency judged only 
after the costs are documented and diarged against the changes in 
student knowledge, judgement, skills, habits, and attitudes that are 
realized 

Two types of data are therefore indispensable in answering the 
question posed in this approach: those derived from comprehensive 
measures of relevant aspects of student performance, and those obtained 
from careful cost accounting of teacher and student time e3q>ended, 
other resources consumed, and undesired side effects induced in the 
educational programme. 



THE METHODOXXX} Y OF CURRENT APPROACHES 

Assessment of perceptions 

The basic data on student satisfaction are variously obtained: some 
institutions employ a standard rating form in which students evaluate an 
instructor or a course, ranking each in relation to others he has known; 
other institutions use a more detailed quesUonmare designed to elicit 
candid reactions to multiple elements (e.g., a particular lecture, the 
required text, tutorial hours, etc.) of the instructional situation With 
respect to each, students may be asked to indicate what they liked or 
disliked and the intensity of that feeling, what was found valuable or 
useless, what experiences they would prefer to have expanded or 
reduced, and the like. Both rating forms and questionnaires are 
often combined with a third type of student reaction form designed to 
probe specific feelings that actions of the instructor have evoked In 
general, such questions (illustrated in Hg. 1) are selected on the basis of 
contemporary learning theory and represent those aspects of the 
student— teacher relationship thought to influence the efficiency and 
effectiveness of learning. 

Data collected from the ratings, questionnaires, and other reaction 
forms are summarized to show general tendencies in student evaluations. 
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FIG. 1. EXCERPTS FROM A STUDENT REACTION FORM 



Did you feel free to challenge the instructor when you thought some- 
thing he said was incorrect? Yes No 

Did you feel free to ask the instructor a question about something you 
did not understand? Yes .. . No . . . 

How often were you stimulated to think about additional applications of 
concepts and principles the instructor was discussing? Practically 
always Occasionally . Rarely Never 

Did you find that the instructor was sufficiently flexible in modifying 
his plans to suit your needs? Yes No — 



When possible, responses are correlated with intellectual and personality 
characteristics so as to enhance their usefulness in faculty efforts to 
indi^ddualize instruction. 

Process assessment 

Broadly speaking, two quite different rationales and associated 
methodologies characterize most forms of process assessment: (1) 
analysis of the conditions for learning samples the elements that 
characterize the total learning situation; (2) interaction analysis focuses 
only on those elements (primarily verbal) that characterize teacher 
beha^dour and/ or student— teacher interaction. 

An analysis of the conditions of learning begms by identi^ng as 
concretely as possible the outcomes sought in a particular instructional 
situation — a course, a lec^ulC, a seminar, a laboratory, ward rounds, 
and the like. For example, the general outcome of ""developing skill in 
physical examination of the patient" could, if specified by a detailed 



I Learning tends to be more efficient when the objectives are clearly 
I understood by the learner. 

Learning of a skill is facilitated by frequent brief practice periods spaced 
over an extended time period. 



Prompt, specific feedback that enables the learner tc identify errors in 
his performance facilitates the learning of a skiM. 

Prompt, specific feedback that reinforces correct elements of a 
periformance facilitates learning. 



FIG. 2. SELECTED PRINCIPLES OF LEARNING 
RELEVANT TO THE LEARNING OF A SKILL 
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definition of the components of that skill, be used as the starting 
point Once the desired outcome has been determined, the next step 
must be to identify the leanung principles that apply most direcdy to 
the specified learning goal* For the illustration ^ven above, the 
principles stated in Fig. 2 would be among those isolated for considera* 
tion. Using such prindples as a frame of reference, an evaluator then 
proceeds to construct a highly structured questionnaire, interview, or 
observation form that will yield objective, descriptive data about the 
extent to which the specified conditions are met in the actual situation. 
An excerpt from such a form, developed for the illustration cited above, 
is shown in Fig. 3. Note that, with minor changes, the questions listed 
there could be employed as a checklist by an observer, as an interview 
schedule by a trained interviewer, or as a structured questionnaire for 
direct distribution to students. In whatever mode they are employed, 
the two essential elements are: (1) that each question be derived from a 

FIG. 3. EXCERPT FROM A FORM DESIGNED TO ASSESS 
THE CONDITIONS FOR LEARNING A SPECIFIC SKILL 



Were the objectives of the course (the instructional unit, the instruc- 
tional exercise) explained to the students? 

j What is the evidence that these objectives were actually understood by 
the students? 

How much opportunity did the students have to practise Mch 
manoeuvre involved In the physical examination of the patient? How 
frequently were these practice periods scheduled? When (in relation 
J to the relevant didactic material) were they scheduled? How long 
I was each practice period? Were variable opportunities provided to 
', . students In accordance with Individual differences in proficiency? 

I How often did the Instructor actually observe the student examining 
I a patient? 

Did the instructor provide any feedback to the student on the accuracy 
of his findings? If yes, when was the feedback provided? In what 
form was it provided? 

! Did the instructor provide any feedback to the student regarding his 
^ technical skill In eliciting these findings? If yes, when was this feed* 
! t^ack provided? In what form was it provided? 



principle of learning that is relevant to the educational outcomes sought; 
and (2) that each question be formulated in a manner that will yield 
objective^ descriptive information^ not opinions or preferences. 

Finally^ it should be noted that, since communication of some kind 
is inherent in any instructional situation, assessment of the conditions for 
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learning wiU also include at least a few generally applicable questions 
that yield data about the basic characteristics of the conununication. 
Tliis evidence would be derived from such questions as, for example: 
"Could the speaker be heard by all parties to the commumcation?"; 
"Did he use terms that were understandable to his listeners?"; "When 
introducing a new term did he define it?"; "Were abstract concepts 
illustrated with concrete examples?**; "Conversely, were specific bits of 
illustrated with concrete examples?"; "Conversely, were specific items 
of information and data related to a general frame of reference?"; 
^'What variety of media was used to supplement the spoken word?''; 
Responses to questions such as these yield the necessary data for 
determining whether the most elementary requirements of the com- 
munication process have been met, and thus whether there is some 
probability that communication has occurred. 

In contrast with the approach described above, the various forms of 
interaction analysis focus exclusively on the nature of the communica- 
tion process. This is accomplished through detailed, systematic analysis 
of the behaviour (predominantly verbal) of teacher and/or Students. 
Originally developed for use in a formal classroom situation, most forms 
of interaction analysis are also useful in other instructional settings. 
Some adherents of this technique (e.g., D. G. Ryans, B. O. Smith^) focus 
attention primarily on teacher behaviour; others (e.g. R.F, , Bales, 
N. A, Flanders and E. J. Amidon^ use die method to examine the 
quantity and quality of interaction between teachers and students. 

Lrespective of the precise emphasis, the general methodology is 
similar for most variants of this approach. Spedfically, observers are 
trained to record at specified intervals (e.g., every 3 seconds), on a form 
(such as that shown in Fig. 4), the nature of the verbal behaviour then 
occurring. These recordings are made for predefined periods (e.g., 
10 — 15 minutes) and the periods are selected to sample different points 
(e.g., the beginning, middle, and end) during an instructional session or 
during several sessions extending over a school term. The recordings are 
then summarized and analysed in terms of such variables as percentage 
of teacher talk as compared with students talk, percentage of teacher 
talk in each behavioua^ category, percentage of student talk in each 
category, characteristic sequences of teacher— student behaviour, and 
the like. 

Certain proponents of this approach define the variables to be 
observed in strictly behavioural terms (as in Fig. 5) in a manner that 
requires the observer to make only minimal interpretation. Others, in 



I Pi Lutfbdft ThcU (1967) The wvalumlon of teaching^ Wiuhlnttoo, D.C«, Baxtta. 
* Afflkloo, E. J. A Hotifh, J. B. (1967) InieractUm aru^ysts: theory, restarch and appltcatUnh 
Keadinr, Mus., Addisoo^msley. 



FIG. 4. EXCERPT FROM A FORM FOR RECORDING 
STUDENT -TEACHER INTERACTION 





t iQina HiA Rvmhnl^ ^hown below make & notfltton 




AUAiv HirAA /'^\ oor^nnrla fnr tnn (10) CORSeCUtivfi 




miniitAQ to InHicfltA thfi nfltiifA of the behaviour 




Amirrtnn at Anrh thre6»SBCOnd Interval 


LEGEND 


Teachtr bthavlour: 




1 = asks a question 












A — rftflArtQ Qtiiriont comment 

■r — iDllvli^lO OlUUUlli wWllllllwlll 




5 = etc. 




Student beha>riour: 








7 = answers a question 




8 = provides Information 




9 — etc. 








10 = silence 




11 = etc. 


Date: 


Hour: Class: . .. . 


Instructor: 


Observer: 


First recording-time : . . 



an attempt to get at more dynamic elements in the interaction and 
to identify potentially facilitative and inhibitory roles played by each 
participant, include such terms as "clarifies", "summarizes", 
"obstructs", "expresses hostility", and "challenges" in the list of 
behaviours that the observer is to record. Inclusion of such terms 
requires more interpretation of the observed phenomena, introduces 
greater subjectivity into the observations, and imposes the need for 
more extended training of observers, in order to achieve acceptable 
levels of inter-observer reliability. However, irrespective of the specific 
categories of teacher and/or student behaviour selected for recording, 
this approach is explicitly limited to observation of the interaction 
process per se; no attempt is made to relate that process to objectives 
or to outcomes, either directly or indirectly through general learning 
theory. 
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Product assessment 



In contrast, product assessment in its pure form is concerned 
exclusively with objectives (product sought) and outcome (product 
realized), and thus requires that the goals of the educational pro- 
gramme be clearly specified, that they be defined in terms of 
behavioural changes to be brought about in the learner, that test 
situations be designed to sample these behaviours, and that these test 
situations be administered to the learner to determine whethei he is 
able to perform in the desired manner and at the prescribed level of 
competence. 

The techniques used to assess performance range from conventional 
tests of information, through sophisticated simulations of clinical 
problems, to long-term, systematic observations of the learner in varied 
professional settings. These techniques, extensively described elsewhere,^ 
do not differ in kind from those regularly employed to certify student 
competence. Only their scheduling and the method of summarizing 
their results need be modified when performance tests are administered 
to students for purposes of teacher, rather than student, evaluation. 
Since in the former case the emphasis is on the change brought about 
in the behaviour of the learner, it is necessary to administer the same 
or comparable tests at the beginning and again at the end of the 
instructional period under scrutiny. Further, if long-term effects are 
included in the objectives of the programme, a complete evaluation 
requires follow-up data regarduig subsequent performance of the 
learners. 

Just as the timing of any assessment is determined by the purpose for 
which it is made, so must the method of summarizing the data be 
dictated by the purpose for which they are collected. In certifying 
student competence it is adequate to report merely whether the student 
does or does not meet the required performance standards. For 
programme evaluation it is also important to know how many students 
do or do not meet each of these standards. Indeed, if large groups 
persistently fail to meet any of them this is a clear sign that something 
is wrong somewhere — in the teaching, the students, the test, or the 
standards. However, for purposes of evaluating teaching such a sum- 
mary is insufficient 

To serve this purpose the performance data must be presented in a 
manner that helps to answer the following questions: (1) How much 
gain was made by the total and by relevant subgroups of students?; 
(2) What are the characteristics of the students who appear to gain 



1 See Charvat, J. ct al, (1968) A review of the nature and uses of examinations In medical 
education, Geneva, World Health Organization (Public Health Papers, No. 36). 
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most (or least) from the current instructional programme?; (3) What 
are the specific areas (e.g., concepts, skills, professional demeanour) of 
greatest and of least gain?; (4) What specific misconceptions (or other 
forms of inadequacy) are still relatively common among students at the 
end of the instructional programme? 

Qearly, conventional student performance reports showing the 
distribution of scores at different points in time jleld very little evidence 
that is useful in answering such questions. Only detailed item analyses 
of the results of each administration of the performance tests will 
suffice. Such analyses can be designed to report the number ot students 
in each group and each relevant subgroup who give each possible 
response and who make each type of error, and tL^ changes over time 
in this performance pattern. Such information is particularly valuable 
when supplemented by various item statistics that report the correlations 
between each response and appropriate criterion variables, and between 
each response and relevant student characteristics anu/or programme or 
instructional variables. Techniques of summarizing performance data in 
this manner are explained in standard texts on test analysis. 

No product evaluation is complete without cost analysis^ particularly 
since evidence is mounting that, given adequate time, personal instruc- 
tion, and self-instructional resources, virtually all students of normal 
intelligence and emotional stability can master almost any element 
among the objectives of higher education. The issue therefore becomes 
an economic one of determining whom a country can afford to educate 
and at what level of mastery, i.e., at what cost in resources. Given tlie 
enuirmous attention currently devoted to the economics of health care 
delivery, it is astonishing how little consideration is beiiig given to the 
economic issues of health professional education. Most presently 
available data are deficient on two counts: (1) they are chiefly 
retrospective; and (2) the data categories are defined in such gross terms 
that it is usually impossible to allocate costs along genumely functional 
lines. 

In order to evaluate teaching and teachers in cost— benefit terms, a 
detailed recording of the amounts of all resources that go into any 
instructional effort is urgently needed. K we are to avoid the notorious 
unreliability of retrospective data, this implies investigating represen- 
tative samples of students and faculty to determine how much of their 
time is actually devoted to some kind of instructional activity, the exact 
nature of that activity, and the way in which the quantity and distribu- 
tion of such activities change with different instructional methodologies. 
It also suggests the importance of costing out other elements in the 
instructional situation, such as, for example, the additional laboratory 
and hospital costs where patients are used as "teaching material". At 



present the only usable data tend to be those concerned with the direct 
money costs attributable to the construction and maintenance of special- 
ized facilities (e.g., lecture halls, student laboratories) and materials 
(educational films, slides, self-instructional progranmies). 

In the absence of hard facts, it is difficult to escape the suspicion 
that, because their costs are hidden, we continue to cling to traditional 
methods that are not only ineffectual but may also be inexcusably 
uneconomic, for they rely on the most costly resources of all — pro- 
fessional time and patient disability. 



USES AND LIMITATIONS OF CURRENT APPROACHES 

Since, if properly interpreted, some data are usually better than no 
data, it could be argued that the primary limitation of all current 
approaches lies in their inadequate usage. However, each one also has 
certain additional shortcommgs that must be assessed if serious errors 
in interpretation are to be minimized. 

Assessment of perceptions — the jallacy that truth is in the eyes of the 
beholder 

Qearly, when perceptions are used as the primary data for evaluating 
teachers or teaching, there is always the danger that student reactions 
will be interpreted as if they were statements of facts, rather than highly 
subjective, iniiividual, and variable opinions. Thus, when a student says: 
"The programme was very valuable; I learned a great deal," there is a 
natural tendency to believe this to be so. Unfortunately, there is no 
evidence to support the view that enjoyment, satisfaction, hard work, or 
any other reported "positive" reaction is necessarily correlated with 
learning. On the other hand, there is also no evidence to support the 
view that something distasteful (either medical or educational) is neces- 
sarily "good" for the recipient Nonetheless, provided the data are 
properly interpreted as simple statements of consumer satisfactions and 
dissatisfactions, the assessment of teaching through student perceptions 
can (1) assist in the diagnosis of some causes of inadequate achievement; 
and (2) contribute to the identification and creation of conditions that 
will enhance motivational and other affective responses that facilitate 
learning. 

Assessment of process — the fallacy that fads and fetishes produce 
results 

The history of therapeutics is replete with examples of interventions 
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earnestly advocated and consistently employed by one generation of 
physicians, only to be proved useless or even harmful by a subsequent 
generation. Widespread adoption of the current "treatment of choice", 
in the absence of suppcMtive data about its relative efficacy, epitomizes 
the pitfalls of a preoccupation with process assessment, independent of 
product assessment For, like therapeutics, education is a victim of 
process fads. 

While educational science has been successful in demonstrating that 
teachers are intolerably costly and inexcusably ineflBcient purveyors of . 
information, or dictators of concepts more accurately recorded in readily 
available literature, it has unfortunately not been so successful in 
defining the teacher's role positively. Repeated experimental studies of 
the relative effectiveness of varied instructional methodologies — 
lecture, discussion, programmed instruction — have produced conflict- 
ing evidence and inconclusive results. It may have been naive to expect 
any definitive outcome from experiments that, on reflection, appear to 
represent a simplistic search for an educational panacea — a pursuit no 
less futile than the quest for an all-purpose, miracle drug. 

In rejecting the role of lecturer and accepting the view that mature 
students are responsible for their own learning, however, some teachers 
have gone through what can only be described as an "identity crisis". 
In this anxious state many have abdicated aU responsibUity for defining 
objectives, planning the educational programmes, creating appropriately 
stimulating conditions and adequately varied resources, setting standards 
of student performance, and gathering the requisite data for ensuring 
that students meet those standards. Regrettably, there has been little 
evidence that adherents of the newer and more sophisticated forms of 
process assessment have attempted to solve this identity crisis by 
utilizing their potentiaUy powerful tools in systematic studies of the 
relation between teacher behaviour and student achievement 

Until the basic connexions between process and product are estab- 
lished, the utility of process assessment must remain limited to research, 
training, and diagnostic applications. As a research tool, process 
analysis opens up the opportunity of determining the relative 
effectiveness of alternative elements in teaching strategies; as a training 
tool it enables the teacher to hold a mirror to his performance and to 
determine the extent to which his actual conduct was a true reflection 
of his intended strategy; as a diagnostic tool, process assessment, when 
used in conjunction with product assessment, yields data that may be of 
value in suggesting alternative hypotheses to explain actual outcomes 
that fall short of intended outcomes. At this stage of educational 
development, process assessment cannot be used in direct evaluation 
of teachers and teaching effectiveness; it can best be used only in 
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conjunction with other methods to assist teachers in improving their 
performance. 

Product assessment — the fallacy of confusing concurrence with 
causality 

Although product assessment furnishes the only data that are directly 
relevant in the ultimate evaluation of teaching effectiveness, it also 
presents the greatest diflSculties in fundamental study design, for four 
reasons: (1) all the factors that normally confound any research on 
human subjects are present m all studies of educational outcome; (2) 
the study population is never randomly selected and its small size and 
lack of comparability with other populations undergoing alternative 
educational treatments make generalization hazardous, even when 
conventional statistical safeguards have been applied; (3) the teaching 
being evaluated usually constitutes such a small proportion of the total 
educational mfluences operating on the student subjects that it becomes 
almost impossible to establish plausible causal linkages; and (4) the usual 
controlled experiment (even less, the double blind) design is rarely 
feasible and almost never suitable. Indeed, Glaser asserts that the 
tyranny of the controlled experiment has probably done more to retard 
educational progress than any other single influence. He bases this 
conclusion on the observation that the only issues that can be studied by 
this method are so trivial that they are hardly worth the egort, that the 
findings are available only at a point in time when they are useless for 
decision-making, and that the requirements of the design are such that 
the conditions to which the findings apply can rarely be duplicated. 

In addition to these inherent Umitations, product assessment imposes 
one additional hardship on the conscientious evaluator: the necessity of 
estimating all elements of the product that are relevant to a particular 
instructional programme. Addre.ssing a very limited range of objectives 
(e.g., increase in factual knowledge) continues to be one of the most 
common failings of studies utilizing this approach. 

Given such severe limitations, it may appear that product assessment 
is of little practical utility. However, at the same time, it is the one 
indispensable approach to the evaluation of teaching. This apparent 
paradox is readily explained: although it may be impossible to establish 
by this method that a given strategy is responsible for success, it is the 
only method by which failure to achieve the goals of teaching can be 
documented and the warning clearly sounded that "something is 
wrong". Other approaches can then be employed in the effort to identify 
and remedy the deficiency. 




CONCLUSION 



Dissonance between the results yielded in the assessment of product 
and those obtained from the assessment of process and perceptions is 
sufficiently common to impose the necessity of collecting a variety of 
types of data as a basis for making inescapable decisions about the 
effectiveness of teaching. Three approaches to the evaluation of 
teaching have therefore been described as sources of the reqmred data 
The methodology of each has been presented, together with a 
discussion of its utility and its characteristic limitations. In the light 
of these limitations a combination of all three approaches is recom- 
mended as indispensable. 
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ADDITIONAL READING 




MICROTEACHING FOR TEACHER TRAINING 
Richard P. Foley^ 



In response to student dissatisfaction with traditional modes of 
training, medical educators everywhere have been engaged in thoughtful 
examination of faculty training programmes, since it has become 
increasingly evident that more systematic and economical methods are 
needed for evident teachers to acquire increased pedagogic skill. 

One alternative to present teacher-training methods that has been 
widely applied by educators &om other disciplines is microieacfung. It 
is the purpose of this paper to describe the technique conceptually and 
operationally, and to offer suggestions for the incorporation of the 
metliod into the programmes of schools for the health professions. 

MICROTEACHING: A DESCRIPTION 

A graduate student in microbiology holds up an agar plate before 
foiu: first-year medical students to demonstrate its use in isolating 
bacterial colonies. Fifteen feet away, a professor of microbiology who 
is supervising the teacher aims the camera of a videotape recorder at 
the group and occasionally writes down some notes. Although the 
teaching lasts for only 10 minutes, two significant things happen. The 
medical students have had an opportunity to learn one method for 
isolating bacterial colonies, and the teacher has had an opportunity to 
practise the instructional technique of presenting a laboratory demon- 
stration. After the presentation is concluded, the supervisor asks each 
student to fill in a questionnaire regarding iht teacher's performance. 
The students then leave the room and a IS-minute critique follows in 
which the supervisor and instructor discuss the demonstration, review 



^ CcQter for Hducationa) Development, University of Illinois College of Medldne, Chicago, 
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the written comments of both supervisor and students^ and view 
segments of the videotape. After a short break, during which the 
practice teacher has the opportunity to incorporate suggestions from the 
critique, the entire process is repeated with a different group of medical 
students. The teacher, the supervisor, and the students have all been 
involved in a microteaching exerdse. 

Microteaching is designed for teachers who want to improve thdr 
teaching skills. The key elements of this trmning method, illustrated in 
the above example, include: 

■ (1) A controlled, constructed setting for producing instructional 
skills. By reducing the length of the presentation, the scope of the 
content, and the number of students, the graduate student in micro- 
biology can practise an instructional skill, such as ^ving a laboratory 
demonstration, in a setting free of many of the complexities of the 
actual classroom. 

(2) Feedback from multiple sources. After ^ving the 10-minute 
microlesson the teachmg assistant is provided with suggestions or 
feedback for improving his performance from a content expert acting 
as the supervisor, from the students he has taught, and from a videotape 
playback. 

(3) The opportunity for immediate integration and application of 
feedback. After the feedback has been received there is an opportunity 
to integrate and apply these suggestions in reteaching to a different 
group of medical students. 

Hve essential propositions underlie microteaching as a training 
method: 

i. The most effective way to master the skills of teaching is to practise 
one skill at a time 

Too often a teacher may be informed by his department chairman, 
students, or colleagues tliat he needs to ''liven up tus lectures'' or ^-aUow 
students more opportunity to participate". Such advice is rarely hdpful, 
for if the teacher knew how to deliver an exciting lecture or increase 
student involvement he would probably be doing so. 

In microteaching, an individual wudiing to improve his instructional 
effectiveness concentrates on the objective of practising a single instrdc^ 
tional skill, mastery of a unique set of curricular materials, or 
demonstration of one teaching method Focusing on specific objectives, 
rather than the teaching-learning process in general, allows a teacher tb 
discriminate among the elements of teaching, and assists him in 
practising a particular technique and incorporating it in his behaviour. 
For example, if the objective of a health educator were to increase 



student participation in the classroom, he mi^t concentrate on udng 
potitive verbal reinforcement (such encouragoig comments as ^Good*\ 
""Excellent^ or other statements indicating satisfaction with the 
response); or non-verbal reinforcement (such actions as nodding his 
head aflfinnativeiy» smiling, ot moving towards the student). 

By concentrating on a particular skill, a teacher can be^ to recognize 
the behavioural components of ^t technique and the situational factors 
tiiat dictate their use. For example* the health personnel educator who 
has learned what kinds of verbal and non-verbal cues constitute 
reintofcement and how to use them can then employ this skill to bring 
about maxunum participation* When a teacher has mastered a variety 
of instructional techniques and knows the effect of each, he can apply 
diem to achieve instructiraal aims in a planned rather than a chance 
arrangement 

2. The complexities of the actual classroom situation limit its value in 
the training of teachers 

Most teachers learn how to teach on the job> but classroom practice 
has significant limitations. First, there is simply too little opportunity 
for trial and error. A bepnning teacher> in particui&r, is often anxious 
about making a good impression on bis students. Consequently, if his 
first lecture is a success and his first group discussion a failure^ the 
chances are that he will become committed to the i ;cture method of 
instruction as the most effective way to teach. Secondly^ ilicru is only 
limited opportunity for a teacher to receive feedback on his perf armance 
in the dassroom, and even that is rarely sought Finally, ami !nost 
important, students are in a classroom to be taught effectively, not to 
be practised on. 

Microteaching is designed to provide a safe setting for the acquisition 
or refinement of educational skills and technique. A be^nning teacher 
mi^t find deUvering a lengthy lecture to a large body of students an 
awesome task. Practising this method of instruction in a microteaching 
setting, where the class size, the time, and the scope of content are 
reduced, c(msiderably alleviates this anxiety. Rather than deliver a 
SO-minute presentation to a group of 75 students, he can concentrate on 
one aspect of the lecture, perhaps the introduction only, and practise 
this component in a S-lO-minute microlesson to a group of 4-5 students. 
Having mastered the introduction, he is ready to practise the body of 
the lecture and the conclusion. Since the students observing the lesson 
would usually not be the teacher's, he is freer to experiment and less 
anxious about receiving student feedback. 
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J. The more knowledge a teacher has of his performance during a 
practice session, the more valuable the experience 

Supervision is generally equated with evaluation and is therefore 
regarded by many teachers as something to be avoided This is 
unfortunate, since constructive superviaon can be one of the best 
resources in a teacher's professional development 

Microteaching greatly increases the opportunity for feedback* Since 
the emphasis is on practising a specific skill or achieving some other 
precise objective, the teacher, the supervisor, and the students have a 
clear picture of the purpose of the microlesson before it is presented 
As a result, the critique that follows is concerned not with die overall 
performance, but with only one aspect of teaching behaviour. This 
approach is generally less threatening and results in more specific 
recommendations for improving teaching. 

The several sources of feedback are deagncd to provide a teacher 
with maximum insight into his performance in the light of his goals. 
The teacher and the supervisor together review student comments, as 
well as video or audio playbacks, and analyse the way in which the 
performance might be further unproved. Ihis feedback can then be 
immediately translated into practice by reteaching the lesson to a 
different group of students. The emphasis in microteaching is on helping 
a teacher to improve, not on judging his performance. 

4. A training method that can be varied is superior to one that is 
rigid 

The most prevalent model for training beguming teachers takes the 
form of education courses supplemented by practice teaching in which 
they observe and work with highly competent "master teachers'', with 
the expectation that practice teaching will help the student's transition 
from the theory to the "real world" of teaching. 

This model has grave limitations. Its value for the trainee depends 
both upon the expertise of the master teacher and his relationship with 
the trainee. The supervisor simply may not allow much opportunity 
foi* the be^nning teacher to practise, or may not provide him with 
adequate supervision. Even in the ideal relationsWp, however, there are 
drawbacks. For example, the master teacher cannot interrupt the class- 
room session to offer constructive suggestions, and there is no 
opportunity for the trainee to stop Ws presentation and repeat some 
portion that needs improvement Also, practice in the classroom must 
be integrated into the flow of the subject-matter: the skili or technique 
being practised must fit well wiUi the content specified for that day. 

Microteaching, on the other hand, allows for maximum control of the 
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practice sessimL The length of the teachings the number and kinds of 
students, and the method of feedback can be varied according to the 
goals of the particular session. For example, the teacher, referred to 
earlier, who is interested in improving his skills in lecturing, can, after 
gaining comfort in presenting a 10*minute lecture to 4—5 students, gLve a 
20-nmiute lecture to 10—12 students, thus more closely approximating 
his teaching in the actual classroom* Unlike conventional practice 
teaching, microteaching provides and opportunity' to pracdse the sidll of 
lecturing until it is mastered, using videotape and mitten comments 
from the supervisor as a record of progress. 

Since a number of variables can be easily manipulated in this 
constructed setting, microteaching is also an excellent research method. 
By manipulating a singjle variable, such as the nature of the supervision, 
countless hypotheses related to the improvement of teacher effectiveness 
can be investigated ^deotapes and the mitten feedback supply the 
researcher with recorded data from which he can work. 

J. A genuine teaching experience is superior io an artificial one 

Peer teaching, role playing, and other simulated teaching experiences 
are being widdy applied in teacher-training programmes. Althou^ 
these methods do offer better opportunities for teachers to practise skills 
than the classroom, there has been a good deal of questioning about the 
transfer of learning from these e^riences to the actual teaching 
situation. Fundamental to the microteaching concept is the fact that it is 
real Although the setting is constructed, bona fide teaching of real 
students occurs. 



mCORFORATING MICROTEACHING 
INTO TKE PROGRAMME OF A SCHOOL FOR THE HEALTH PROFESSIONS 

Health personnel educators i^ce the dual task of improving the 
instructional skills of existing faculties in schools for the health 
professions and of developing these skills in graduate students about to 
begin teaching. Perhaps one of the soundest and most economical routes 
for achieving both ends is to train the two groups simultaneously, using 
microteaching as the vehicle. 

Where to begin 

Graduate students in both basic and clinical sdences frequently 
function as teachers under an experioiced faculty member as part of 
their academic work, but rarely does any formal training in education 
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accompany this txpeneocc. This situation provides an excellent 
opportoni^ to introduce microteachmg as a method for as^ting these 
students to acquire the necessary instructional sidlls. Short-term 
imgrammes could be designed to increase thdr competence in 
conducting lectures, demonstrations, or ffoap discussions. Since these 
programmes would not necessarily have to constitute a forma!, academic 
course, both established faculty and graduate students mig^t be ofiEered 
this opportunity. Hie foculty members could be asked to present a live 
or videotaped lecture or laboratory demonstration to serve as a model 
fcr introdudng the graduate student to these techniques, to function as 
supervisors in microteaching sessions, or to serve as consultants in 
deigning the training programmes. 

Designing and implementing a microteaching training programme 

One of the most important decidons in dedgning a microteaclung 
programme is identification of the specific techniques or strategies to be 
practised by the participants. Snce training in instructional skills that 
are not relevant to trainee needs is generally wasted effc^ it is {mident 
to involve potential participants in the planning stage throu^ interviews 
or questionnaires that can quickly isolate those instructional areas in 
which they feel some need for assistance. 

An expert in education is needed to assist participants in this 
identification, to provide the educational content of the programme, and 
to oversee the operational details of the microteaching setting. For a 
small programme with 15—20 trainees, one educator and 3—4 faculty 
supervisee familiar with the content of the programme and trained in 
the microteaching process are suffident If videotape recciders are 
used, an operator must be available to man the equipment If the 
supervisor can be spared this task, it permits him more time to concen- 
trate on the teacher's performance. If not, sq)ervisors can be quickly 
trained to operate the equipment For a larger training programme, 
staff should be increased accordin^y. 

Althou^ several options are available for introducing this training 
activity to participating teachers, it will be most productive if at least 
one day before the microteachmg programme htgrn partidpants read 
the material that describes the skill to be [vactised; take part in a 
discusaon that leads to the formulation of those criteria that 
demonstrate effective use of the skill; view a live or videotaped 
presentation of the skill to which they can BpjAy the criteria; design 
the evaluation forms to be used by students and supervisors; plan a 
microlesson that incorporates the skill to be practised; and become 
famiUar with such operational det^ as the expected length of their 
lesson, the time of the reteach, and tiie kinds of students to be taught 
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Since microteaching is real teaching, students must be recruited vAio 
approximate, as nearly as possible, to those that the teacher ordinarily 
encounters. If, for example, one is conducting a microteaching 
programme to improve the questiomng skills of teachers responsible 
for a basic science programme in a medical school, then beginning 
medical students would be the most suitable choice. 

Although the number in a microteaching session can range ficom 
1 to 10 students, depending upon the purpose of the lesson, 4 or S 
students is usuaL In deciding upon the number it is well to keep in 
mind that microteaching is dedgned to assist teachers, not students. 
The setting, therefore, should be one in which teachers feel free to 
experiment without fear of failure. Unless a large number of students 
can be justified, the groups should be kept small so that concentration 
of the skill and maximum feedback can be achieved. 

There is no prescribed length for the microlessons; they could vary 
in length from 3 to 30 minutes, depending upon the goals of the 
programme, the number of participants, and the time available. The 
most frequent length is 5— 10 minutes, since this period generally affords 
adequate opportunity to practise a particular skill and to cover a 
significant amount of content 

Similarly, the length of the critique may vary. Ten minutes is 
generally adequate to aUow feedback from students, supervisors, and 
videotape. With the format of a 5-minute teaching session and 10-minute 
critique, two teachers can complete a teach-critique-reteach-critique 
cycle in an hour. For example: 



9.00-9.05 


teacher "A"— teach 


9.05-9.15 


critique 


9.15-9^ 


teacher "B"— teach 


9^0-930 


critique 


9J0.9J5 


teacher "A" — reteach 


9J5-9.45 


critique 


9.45-9^0 


teacher "B" — reteach 


9JO-10.00 


critique 



Although the length of time between the teach and the reteach can 
be varied, if the reteach is gjiven immediately after the first critique 
there is generally not suJOBcient time for the teacher to incorporate in 
reteaching the suggestions offered. On the other hand, reteaching a day 
or two later weakens the effect of the feedback received from the 
critique. An mterval of 15 minutes to an hour between the teach and the 
reteach enables a teacher to plan a second lesson with the feedback 
suggestions freshly in mind. 

Super/ision is of critical importance to successful microteaching and 
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merits the following considerations* First, as the focus of the critique 
increases, its effectiveness decreases. It more than two suggestions are 
offered for chan^g performance, a teacher often becomes confused, 
defensive, or discouraged. Secondly, supervision should be supportive 
and lead the trainee towards self-diagnosis. This can best be achieved 
by allowing time for the subject to comment upon his own performance. 
The supervisor can assist by asking such questions as: "What did you 
tlunk of the lesson?"; "Do you feel tiiat you met tiie goals you set out 
to accomplish?"; "What will you do differenfly in your reteach?"* If 

EVALUATION SHEET: SELF^ANALYSfS LESSON 



Teacher _ Date 

Instructions; Complete questions 1 and 2 before your first fos$on. 

Complete questions 3 and 4 t}efor8 your reteach lesson. 

Complete question 5 after your reteach lesson. 

Take this completed form with you to the conference with 
your supervisor. 

1. List the objecthres of your lesson. 



2. How do you intend to accomplish these objectives? 



3. How successful was the lesson? Cite positive and negative aspects. 



4. What changes do you intend to make for the reteach lesson? 



5. How affective were the changes you made from the first lesson? 
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the teacher identifies his own problems, he is much more likely to seek 
and reach solutions that will lead to improvement of that performance. 
Thirdly, if videotape replays are available, the supervisor should use 
this opportmiity to reinforce effective behaviour, as well as to point out 
dcfidences. Whenever appropriate, the tape should be stopped, com- 
ments given, and the segment replayed Deferring comments until the 
end of the replay minimizes the reinforcement potential of the 
experience. 

Finally, evaluation forms provide another valuable source of 
feedback* They can be used to record reactions from students, super- 
visors, peers, and the teacher himself. A sample form that a teacher 
might use in analysing his own performance is reproduced in the 
accompanying figure. The question could obviously be reworked into 
reaction forms appropriate to students and supervisors. The most 
appropriate evaluation forms, however, are those designed to respond 
to the specific objectives of the microlesson and the individuals 
responsible for evaluation. 



CONCLUSION 

Microteaching should not be viewed as a panacea for all the problems 
related to the training of teachers. While it may, for example, help a 
teacher to become more conscious of his behaviour, it cannot transform 
a dull teacher into an intellectually exciting one. Its great advantage is 
that several valuable training functions occur simultaneously: it provides 
a setting that focuses on specific skills, enriches supe^^^sion, facilitates 
continuous training and research, and offers safe but realistic practice in 
instructional techniques. 

Above ull, microteaching is a flexible teacher-training method and 
should not be abandoned if, for example, videotape equipment or 
experienced supervisors are unavailable. Since these and other variables 
can be adjusted, a precise definition of the microteaching context can 
vary according to the objectives desired and resources available. 
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THE IMPLEMENTATION OF WHO'S TRAINING 
PROGRAMME FOR TEACHERS OF 
THE HEALTH PROFESSIONS 



TamdsFulop^ 



The background and content of WHO's comprehensive, coordinated 
long-term programme for teachers of medical and allied health sciences 
have been described elsewhere.^ This paper is concerned with the 
implementation of that programme, the purpose of which is to 
contribute to the improvement in the delivery of both preventive and 
curative health care. 

A Memorandum of Agreement was signed between WHO and the 
University of Dlinois, Chicago, DI., USA, on 30 September 1970. It 
states in its preamble that 'It is the desire and intention of the parties 
to bring about a comprehensive and co-ordinated training programme 
for teachers from schools of medicine and allied health sciences which 
wili enable the participating WHO Regions to set up re^onal and local 
(intercountry and country) Health Personnel Teacher Traming Centres 
and thereby contributing to improve not only the teaching standards 
of schookof health personnel, but also to meet the quantitati^^e needs* 
i.e., manpower planning in relation to teaching stafP*. The operational 
paragraphs state, inter aliay that: 

(1) The University shall provide ia tacb of the academic years 1970/71 
through 1973/74 within and by its Center for Educational DevelciMnent various 
training programmes and courses for selected candidates from each of WHO's 
participating Regions, 

(2) ... the training programmes and courses shall include the following: 

(a) A programme of one-year duration for two to four fellows each academic 
year which may lead to a degree in Medical (Health Personnel) Education; 

(b) One course (seminar/workshop) of four weeks duration each academic 
year for 12 to 14 participants per course from WHO*s participating Re^ons; 



^ Director, Division of HealUi Manpower Development, World HealUi Organization, Geneva, 
Switzerland. 

* FOIdp, T. (1973) Training teachers of health ptrsonnel. To: Development of eduauionat 
protrmmmes for the health professions, Geneva, World HealUi Organization ij^ublie Health 
Papers, No. 52), p. 84. 




(c) One two to four week workshop on-site in the Regions each acaciemic 
year for a maximum of 25 participants. 

The one*year course is designed to train the future full-time members 
of Regional Teacher Training Centres (RTTCs), among them the 
directors and deputy-directors. The 4-week seminar/workshops are 
intended for those who will become part-time teachers for the RTTCs* 
Finally, the 2-week on-site workshops are designed to "prepare the 
ground" for the RTTCs by discussing relevant problems with the 
teaching staff of the institution where the RTTCs will be located 

The goals and objectives for these three types of training have been 
properly defined (see Annex). The training activities within the frame* 
work of this Agreement started in June 1971, when the Center for 
Educational Development (CED), University of Illinois College of 
Medicine, Chicago, started to function as an Interregional Teacher 
Training Centre for Schools of Health Personnel (IRTTQ. 

In 1971 the WHO Regional Offices for Africa, the Eastern Mediter- 
ranean, South-East Asia, and the Western Pacific proceeded to select 
sites for RTTCs. First the Eastern Mediterranean, then the Western 
Pacific, and finally the South-East Asia regional offices called upon 
consultants to assist in the selection of a site and in drawing up a plan 
of operations. The site of the P.TTC for the African Region was 
selected by the regional office staff. 

Consultant visits were arranged in Shiraz (Iran), Sydney (Australia), 
Bangkok (Thailand), and Peradeniya (Sri Lanka). For the purpose of 
these feasibility studies, criteria were elaborated and adopted by the 
consultant groups to standardize procedures for site selection. These 
criteria were intended to be used as a guide in the data collection and 
subsequent evaluation during consultants* visits. The consultants also 
drew up recommendations for the case where the site of an RTTC is 
selected by the regional office concerned. 

In 1971-1973 the regional offices proceded with the implementation 
of the programmes as follows: 

The WHO Regional Office for Africa assisted in the establishment of 
an English-language RTTC at Makerere University, Kampala (Uganda) 
and of a French-language RTTC at the University Centre of Health 
Sciences, Yaounde (Cameroon). 

The WHO Regional Office for the Americas assisted in the establish- 
ment of a Latin American Centre for Educational Technology in 
Health Sciences in Mexico City (Mexico) and another in Rio de Janeiro 
(Brazil), which perform the functions of an RTTC. 

The WHO Regional Office for the Eastern Mediterranean assisted 
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in the establisment of an RTTC at the Pahlavi University Medical 
School, Shiraz (Iran). 

The Regional Office for Europe, considering teacher training to be a 
high-pri(»ity matter, planned activities such as educational meetings. 

The WHO Regional Office for South-East Asia assdsted in the 
establisment of regional centres at the Faculty of Medicine, Chula- 
longkom University, Bangkok (Thailand) and at the Faculty of 
Medicine, University of Sri Lanka, Peradeniya (Sri Lanka). 

The WHO Regional Office for the Western Pacific assisted in the 
establishment of an RTTC at the University of New South Wales 
Medical Faculty, Sydney (Australia). 

The establishment of National Teachers Training Centres (NTTCs) is 
now being considered in alllhe regions. 

The following training activities have been conducted by the IRTTC 
within the scope of the above-mentioned agreement, with a view to 
preparing future leaders and collaborators of RTCCs, where leaders 
and teachers for NTTCs will later be prepared: three 1-year courses in 
1971/72, 1972/73, and 1973/74, with a total of 15 participants; two 
4-week seminar/worL>hops in 1972 and one in 1973, with a total of 49 
participants; and a number of 2-week on-site workshops, including 
those in Sydney, Shiraz, and Bangikok (on-site workshops were also 
organized in many other places without tiie assistance of the IRTTQ. 

An Interregional Seminar on Teacher Training for Schools of 
Medicine and Allied Healdi Sciences was held at tiie IRTTC in 1971, 
with 12 participants, including top-level administrators of schools for 
healUi personnel where RTTCs or NTTCs may be established. The 
purpose of this seminar was to ^ve participants a greater insight into 
the nature of educational leader^p and the requirements for 
facilitating the implementation both of basic educational programmes 
for the health professions and of programmes designed to train 
teachers for the health professions. A similar seminar in the French 
kmguage was held in Yaounde (Cameroon) in 1972. 

The network of RTTCs in those WHO regions wishing to establish 
tiiem was completed by the end of 1973. The first NTTCs started flieir 
activities in 1974, and all Member States wishing to establish such 
centres may have at least one by the end of this decade. Centres for 
educational research and development at the level of individual schools 
have ahcady started to develop in several countries.^ A mo/e rapid 
development may be expected in the second half of this decade after 
the NTTCs have started working. The activity of th* c centres in the 



^ See Educailonal rtsearch and development centres for the health professions (uspubltihed 
document WHO/BDUC/74.172). 
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schools in the field of training, research, and service is, of course, not 
an end in itself. Their main purpose is to assist the schools in training 
health workers who will meet the real health needs and demands of 
the community they will later serve. 

Now, a further development is being considered — teacher training 
centres to serve as nuclei for the establishment of so-called health 
manpower development centres (HMDCs).^ These would be structures 
within the framework of which the three main elements of the health 
manpower delivery process— rthe planning, "production", and sur- 
veillance of health personnel — ^would be integrated. Tbtse three main 
elements are mostly being developed separately in a somewhat isolated 
way, and their development has reached different stages m different 
countries. The planning of health manpower, i.e., delSning the quality 
and quantity o? manpower needed for the eflScient coverage by health 
services of tl' e entire population, is being developed in many countries, 
but in most cases it has no or hardly any influence on the second 
element, the "production*' of manpower. This means that health man- 
power prouuction is all too often isolated from the planning process 
and thus &om the health ser^ces for which it is supposed to cater, 
with the result that the manpower that is trained is not always 
appropriate to the country's needs. The third element, the monitoring 
of health manpower, i.e., the control of whether or not the trained 
health personnel are able and ready to cope with the tasks for which 
they were trained and to feed the data back into the planning and 
production phases so that any necessary corrections may be made (to 
close the cybernetic cycle of the manpower process), is hard to find 
anywhere. 

The mission of the planned HMDCs would be to promote the 
balanced development of these elements, to ensure that the health 
manpower process fulfils its main task, namely the provision and 
maintenance of manpower for the total health coverage of the entire 
population by health personnel who are motivated and able to serve 
their community* 



^ Such centres do not exist anywhere yet, as they tre only now (1974) being planned. 
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GOALS AND OBJECTIVES FOR EDUCATIONAL ACnVITIES 
TO BE CARRIED OUT BY 
THE CENTER FOR EDUCATIONAL DEVELOPMENT, 
UNIVERSITY OF ILLINOIS COLLEGE OF MEDICINE, 
CHICAGO, ILLINOIS, 
WITHIN THE FRAMEWORK 
OF THE WHO TEACHER TRAINING PROGRAMME 



ONE-VEAR FELLOWSHIP PROGRAMME 

A fellowship year at the Center for Educational Development provides 
a setting for an individual to define his role as an educational leader in the 
health professions. Some may elect to move towards this goal by enrolling 
in the Graduate Programme leading to a Master's degree in education, 
others by engaging in more focused work on a specific project. Whether 
through formal courses, independent study, or project work, each Fellow 
will be provided with the opportunity to develop skills in such areas as 
educational planning, educational programme implementation, and educat- 
ional research and evaluation. 

Educational planning 

Fellows will be afforded opportunities to engage in several kinds of 
educational planning durng the course of a year, including short units 
of instruction (e.g., 1-3 hours), short-term training programmes (e.g., 1~4 
weeks) for health personnel, or instructional programmes in the biomedical 
discipline from which they come. Participants may expect to acquire such 
skills essential for effective planning as: 

— deriving objectives from multiple sources 

— defining programme goals 

— formulating objectives in terms of expected student outajmes 

— using generally accepted principles of adult learning to develop strategies 
for implementing planned programmes 

— specifying concrete, manageable evaluation strategies 

— articulating alternative techniques to achieving specified objectives, 
emphasizing the advantages and disadvantages of each 

— facilitating the active involvement of educational specialists when 
appropriate 

— identifying and utilizing varied kinds of educational resources 

— identifying potential sources of funding and procedures for acquiring 
them. 

Programme implementation 

In addition to increasing their competence in programme planning, 
participants will also be afforded multiple opportunities to acquire some 
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of the skiUs necessary for effective programme implementation. During the 
year they can expect to increase their competence m some or all of the 
following areas: 

— translating learning objectives into realistic instructional sequences 

— stimulating faculty and student enthusiasm and capturing commitment 
to a programme plan 

— facilitating the constructive resolution of interpersonal conflicts that 
interfere with the effective implementation of planned programmes 

— recognizing the advantages and disadvantages, as well as skilful use, of 
such methods as the lecture or small-group discussion 

— recognizing the strengths and limitations of such instructional media as 
films, audiotapes, television, and skilful incorporation of educational 
technology into instructional efforts 

— facilitating the professional growth of colleagues as teachers 

— assessment of personal as well as programme strengths and weaknesses 

— systematic experimentation with both organizational and personal imple- 
mentation strategies. 

Research and evaluation 

The focus of the fellowship programme will be upon the acquisition of 
those research and evaluation skills that can aid in the conduct of action 
research rather than in basic investigations. While it is unrealistic to 
expect that participants will become sophisticated investigators, the skills 
they acquire should make it possible for them to assess the major elements 
of student achievement and programme effectiveness. This capability should 
be manifested as increased proficiency in some of the following areas: 

— utilizing formative and summative evaluation procedures 

— selecting evaluation techniques that will measure at different levels: 

(a) cognitive processes 

(b) attitudes, interests, motivations 

(c) skills 

— constructing performance tests appropriate to such objectives 

— using and interpreting standardized evaluation instruments 

— interpreting evaluation data 

— formulating a research problem 

— utilizing work reported in the literature to identify areas for further 
study in medical education 

— using basic statistical techniques (e.g., probability, parametric and non- 
parametric statistics; 

— writing reports for publication. 




FOUR-WEEK seminar/ WORKSHOP TEACHER TRAINING PROGRAMMES 

The general purpose of the 4-weck workshop is to provide participants 
with an opportunity to refine educational leadership skills that can be 
applied to the educational problems encountered in the home setting. 
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The intention is not that they will leave merely with an increased store of 
information about the educational process but rather that they will be 
better prepared to use both old and u^wly acquired knowledge in a manner 
that will: 

— sdmulate local improvements in educational practices 

— provide a personal example of the application of educational principles 
to teaching activities 

— allow them to teach colleagues some of what they have learned. 
More specifically, attention will be directed towards five broad areas 

that represent the core of effective educational planning and implementa- 
tion: 

(1) The nature of learning 

— principles of adult learning 

— application to curriculum and instructional planning 

— experimental set 

(2) Curriculum development 

— the nature, source, and specification of educational objectives 

— influence of social forces upon programme planning 

— rationale for specific planning system 

— design of programme 

(3) The nature of teaching 

— selection of instructional methods 

— development of greater skill in selected techniques (e.g., lecture, 
group discussion) 

— sensitivity to individual and group needs 

(4) Instructional media 

— strengths and weaknesses of basic communication devices (e.g., 
slides, films^ television, audiotapes) 

— selection of media 

— skill in preparation of simple media (e.g., 2 mm X 2 mm slides, 
audiotapes, transparencies) 

— more skilful use of selected media 

(5) Evaluation 

— principles of student, teacher, or programme evaluation 

— strengths and weaknesses of specific testing methods (e.g., oral, 
essay, multiple-choice, practical examination) 

— assessment of problem-solving, clinical performance, attitudes 

— test scoring and reporting 

— use of tests as teaching tools. 

The programme is organized so as to allow each participant an 
opportunity to pursue in greater detail a special interest in one of these 
areas and will require all participants to develop at least one instructional 
unit that he can take back to his home setting and share ^th colleagues. 
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The 4 weeks will include: 

— 3 sessions dealing with learning 

— 10 sessions dealing with curriculum (of which 2 will specifically relate 
to objectives) 

— 18 sessions dealing with teaching (including 5 half-days specifically 
devoted to microteaching, during which each participant will have an 
opportunity to teach his special instructional unit to others) 

— 10 sessions dealing with instructional media 

— 10 sessions dealing with evaluation principles and techniques. 

In addition, time will be set aside for individual reading, r^^'onal con* 
suUation with staff members, and participation in colloquia at which 
distinguished visitors will serve as discussion leaders. 



WHO ON-SITE WORKSHOPS 

General goal 

At the conclusion of a 2-wcck on-site programme each participant 
should be able to demonstrate that he comprehends the elementary com* 
ponents of a sound educational process, that he can plan the application 
of such general principles to solve specific educational problems in his 
own setting, that he can use at least one instructional or evaluation 
technique with greater skill than prior to the workshop, and that he 
appreciates the importance of systematic educational studies. 

Specific objectives 

Achievement of the general goals will be approached through the 
.following activities in three broad areas: 

(1) Educational programme planning 

— identify major sources of programme objectives, from within an 
institution and from outside 

— plan a rational system for the review and selection of institutional 
and departmental goals 

— identify conflicts in determining goals and priorities and suggest 
mechanisms for their resolution 

— recognize ways in which one's own actions may facilitate or impede 
the planning process 

— i-e/>/^««izc the implications for teacher performance of a faculty 
Commitment to independent student study. 

(2) Instructional strategies 

— identify the major elements of adult learning 

— recognize teaching practices that facilitate and impede learning 

— distinguish among cognitive, psychomotor* and affective learning 
objectives 

— write behavioural objectives in each of these domains 
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— justify the selection of specific instructional methods (either old 
or new) in terms of the objectives they are intended to serve 

— develop a preliminary plan for an instructional imit, with dearly 
stated goals and appropriate materials and methods 

— identify the cost-benefit issues in connexion with the selection or 
development of audiovisual aids to learning. 

(3) Evaluation tactics 

— define in non-technical terms the concepts of test reliability, 
" validity, norm-related and^criterion-related measures, and forma- 
tive and sununative evaluation 

— describe the several levels of complexity in the taxonomy of 
educational objectives (cognitive domain and affective domain) 

— compare and contrast the relative merits of essay, oral, practical, 
and multiple-choice examinations and such newer procedures as 
PMP, role-playing, and chart review in assessing specific kinds of 
educational goals 

— develop an evaluation instrument for the instructional unit under 
(2) above 

— list criteria upon which teaching effectiveness may be judged. 

Evaluation procedures 

The assessment of progranune impaa will be directed more towards 
general goals than specific objectives since the latter are principally means 
by which the former are to be achieved. The mechanisms for accumulating 
data upon which an evaluative judgment may be made include: 

(1) Preliminary background information. This will include questionnaire 
data about each participants' educational responsibilities, his prior study 
of the educational process, a description of the educational problems he is 
facing, and the means by which he is attempting to solve them. In addition, 
each participant will be asked to prepare a solution to an educational 
workshop experience. 

(2) Performance during the workshop. Staff and participants will 
togeUier develop means of assessing individual and group progress towards 
the specific objectives ultimately accepted by the full group. This ongoing 
assessment will provide continuing feedback designed to influence pro- 
graiiime evolution in a fashion that will optinuze the group's output 

(3) Terminal judgement Participants and staff will, at the conclusion 
of the workshop, independently identify the extent to which specific 
objectives have been achieved. A sharing of these independently generated 
views will serve as a terminal assessment In addition, each participant 
will be asked to summarize what he now intends to do with what he has 
learned and what he expects to achie\'e in the following 6 months. 

(4) Follow-up. Six months after the workshop there will be a follow-up 
of each participant to determine the extent to which he has been able 
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to translate his learning into practice. The questionnaire to generate such 
information will be developed jointly by participants and staff in the 
course of the workshop. 

Special exceptions 

Since each workshop serve a different participant group it can be 
antidpated that both needs and wants will vary. The general goals and 
specific objectives set forth here will serve as the basic guide for most 
programmes but it is essential to preserve options; thus general emphasis, 
as well as specific content, may vary with the needs of different regions. 
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PRINCIPLE INTO PRACTICE 
George E.P>Iiller^ 



Translating into institutional or classroom practices the principles 
suggested in this volume and in the earlier WHO publication^ on the 
subject is not easy. In fact, to accomplish the fundamental system 
changes that must ultimately accompany full implementation of these 
educational strategies will always be arduous, often firustrating, and 
frequently discourag^g worL For this reason the final paper deals 
with perspective, so that those who encounter trying problems, 
seemingly insuperable impediments, disheartening apathy, or outright 
hostility among staff colleagues will not feel entirely alone and can 
see that there is light at the end of the tunnel 

The systematic application of educational science to problems of 
medical education is a relatively recent event It was only 15 years ago 
that the first formal imits of educational research and development were 
established almost simultaneously at three medical schools in the 
United States; the first primitive efforts to bring educationists and 
medical teachers together in prolonged collaboration on common 
problems had been initiated only S years earlier at the University of 
Buffalo School of Medicine. Despite the senunal influence exerted by 
Uie small group Uiat began flie work fliere, this pilot effort left scarcely 
a trace in tiiat institution when the leaders moved on to pursue their 
work elsewhere. To explain this apparent defection one might use a 
biomedical analogy: antigenic stimulus produces an antibody response 
which if sufficientiy intense may lead to rejection of a transplant 

There seems litfle doubt fliat such a scries of events took place in 
Buffalo. Vigorous exploration and espousal of unfamiliar ideas about 



1 Director^ Center for Educational DeveIopment» Ualvenity of lUInols College of Medicine, 

^^^vt^rld^alth Oraanlzatlon (1973) Development of educational protrammts for the health 
professions, Geneva {Public Health Pcpers, No. 52). 
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educational process which also challenged long-standing practices of 
established medical teachers almost inevitably aroused a defence 
reaction on their part And a missionary group of youthful enthusiasts 
was not sufficiently skilful in anticipatmg, preventing^ or dealmg with 
that reaction to dissipate its force. Fortunately,- significant learning 
accompanied that experience, and those vAio moved on were able to 
manage more effectively in other settmgs the human dynamics that are 
predictably set in motion by the introduction of new and unfamiliar 
ideas. Since many who read this volume will encoimter such reactions 
if they attempt to translate educational principles into institutional 
teaching practices the experience of another worker may be comforting, 
and even helpful. 

The University of Illinois College of Medicine, now the site of what 
is probably the most comprehensive unit of educational research and 
development in a health professions school in tiie United States, was 
no more advanced in its educational practices than most other medical 
schools when the Office of Research in Medical Education was 
established there in 1959. The unique features were a Dean who was 
persuaded that systematic study by full-time staff members profes- 
sionally qualified i nedicine and education was essential to sound 
educational progranune planning or implementation, and a group of 
teachers who had thought enough about these issues to have some 
feeling for the dimensions of the problems as well as a willingness to 
attack them in a new way. Aldiough there was no clear and widespread 
understanding of what such a unit would do, the appearance of a 
physician/educator and an educational psychologist was greeted with 
cautious enthusiasm. 

This feeling lasted for nearly a year while these new members of 
the College faculty devoted themselves to the task of asking individual 
teachers, faculty committees, administrators, and students about general 
educational goals, the educational problems th^ were meetmg, and 
what they thought was required to solve them. But the inquiry was 
more than simple data-gathering; it also communicated the kinds of 
things such professionals saw as important issues: the specification of 
objectives, the match between objectives and instructional programmes, 
and the reliability and validity of evaluation procedures in gathering 
data on student achievement and programme effectiveness, for 
example. The anxiety such probing produced was generally tolerable, 
although the educational implications were often harshly discussed. 

Somewhat greater uneasiness was produced among the faculty by a 
second data-gathering procedure that had been tested earlier in the 
Buffalo project direct observation and systematic documentation of the 
teaching/learning interaction. Literally hundreds of hours were spent in 
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lecture halls, classrooms, laboratories, clinics, and wards gaining what 
proved to be a remarkable first-hand picture of the educational 
experiences, as well as the instructional content, to which students were 
exposed 

The scepticism with whidi this work was greeted by teachers who had 
expected action, not merely study by a team of professional educators, 
was further hei^tened when they were asked to authorize or participate 
in specific investigations designed to elicit more detailed information 
.about particular aspects of the educatior.al programme — for example: 
(1) student and faculty attitudes and values; (2) student and faculty 
perceptions of institutional climate (incentives and rewards); (3) student 
problem-solving and reading skills and their influence upon achieve- 
ment; and (4) tiie retention by students and by faculty of knowledge 
gjuned in basic science courses. 

Out of tiie evidence accumulated in these ways emerged conclusions 
that provided a base fDr suggestions, proposals, and recommendations 
that the educational development staff now felt justified in making. 
Unfortunately, but predictably, they were neither the kinds of 
conclusions nor the types of recommendations that most teaching staff 
wanted to hear. Two illustrations may serve to lughlight this 
discrepancy. 

First, one outcome of die study was a clear impression that it was the 
examination system rather than educational objectives, curriculum 
organization, or instructional techniques that had the most profound 
impact upon student learning. For no matter how appealing the state- 
ment of goals, how logical the programme organization, how dazzling 
the teaching methods, it was the examinations that conmiunicated most 
vividly to students what was expected of them. Unfortunately, the 
medical school examinations seemed more often at odds with explicit or 
implicit instructional objectives and programme organization than 
consistent with them. For example, there was widespread agreement 
among the teaching staff on the importance of helping students to 
acquire die skill of identifying, analysing, and managing problems, but 
the examinations tested chiefly their skill in recalling isolated fragments 
of information. And since the examinations wc;e conducted by 
individual departments, while content was often common to several 
departments, the rewards went not only to the students who could 
recall facts fully but also to tiiose who could recall different facts to 
answer the same question depending upon who asked it, e.g., whether 
internist, surgeon, or psychiatrist asked about the management of 
peptic ulcer. Such an observation suggested that real change in the 
educational programme should begin with the examination system not 
with curriculum manipulation. Ideally, such a system should include 
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both a guidance function for students and faculty, in order to facilitate 
learning, and a quality control function, to certify that required learning 
has been achieved Its accomplishment requires more than the sincere 
efforts of teachers working independently in isolated departmental 
enclosures; it demands both institutional organization, with checks and 
balances upon individual or departmental enthusiasm for a particular 
facet of knowledge, and a professional support staff well informed 
about theory and skilled in tiie practice of educational evaluation. 

A second illustration is drawn from the arena of curriculum 
organization. There was general acknowledgment that the body of 
biomedical knowledge is so large and is changing so rapidly that 
"coverage" in the course of a basic programme of medical educatioi 
is impossible. A more important educational goal than covering content 
was that of helping students to learn how to learn mdependendy so that 
tiiey would be prepared to deal with chanpng biomedical knowledge 
througihout tiieir professional career. Nonetheless ciuriculum planmng 
was dominated by tiie batfle for teachmg time in order to provide for 
all students what teachers claimed to be a bare minim um from each 
discipline. But when those bare minimimis were brought together the 
resulting curriculum was so loaded with teaching tiiat students had 
virtually no opportunity to learn how to learn independently. And 
having thus been encouraged to become dependent, it should have come 
as no surprise when they exhibited litfle interest in "keeping up" after 
graduation. The solution to the problem of continuing physician 
education, about which teaching staff were very concerned, seemed to 
lie not so much in providing more postgraduate continuing education 
programmes as in establishmg new patterns of basic medical education 
that truly encouraged and rewarded independent learning. But this 
inevitably meant giving up "teaching tim»^", and few departments were 
ready to take tiiat step, altiiough many were ready to agree tiiat otiier 
departments could easily do so. As in the case of change in the 

examination system, this required more than individual efforts it 

demanded an institutional system of checks and balances in instructional 
programme planning and implementation. 

As such data and ideas were surfaced, facul^ scepticism shifted to 
poorly concealed hostility, for it became clear tiiat tiie educational 
experts imported to assist teaching staff in carrying out their mission 
were not going to be content with merely restructuring the curriculum 
in some more fashionable mode or introducing more varied instructional 
technology; they were advocating fundamental changes in educational 
decision-making as well as teaching practices. And thus tiie batfle was 
joined, as it will be m any medical school tiiat undertakes such basic 
self-scrutiny. The ultimate outcome of this confrontation will depend 
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upon many things, but three seem particularly important if the concepts 
advocated here are to prevail: adminitrative support, faculty involve- 
ment, and the maintenance of a spirit of inquiry. 

The Dean, Vice-President, or Vice-Chancellor need not be expert in 
educational science, but without their understanding and advocacy this 
basic reorientation to programme planning and teaching roles is 
unlikely to survive the direct attacks and indirect sabotage to which 
proposals for significant change in educational practices will be 
subjected* K these leaders are unwilliag to face disgruntled associates 
who protest Invasion of previously inviolate departmental autonomy, cr 
are not convinced that the body of knowledge accumulated by profes- 
sionals in education has clear applicability to health professions 
programmes, then the likelihood of such efforts becoming an integral 
element of institutional programming is very small. 

It is equally small if those attempting to introduce the principles of 
educational science are insensitive to the interests, questions, and 
anxieties of their teaching colleagues or ignore the usual channels of 
communication and authority. If a school is fortunate enougih to have 
a faculty organization and committee structure through which policies 
are established and programmes operated, then working changes with, 
or through, such groups is the optimal path. If a school is 
organizationally no more than a loose federation of essentially 
autonomous departments, then working patiently within this structure 
towards the development of coordinated or cooperative and ultimately 
colle^al decision-making is the most reasonable path. Either of these 
methods is more time-consuming than instituting programme changes 
within a more limited area of personal responsibility. And the easier 
way may provide an overt model of new instructional stratepes or 
tactics for others to see, but it will not necessarily have any impact 
upon the educational practices of others. The real task facmg an agent 
of educational change is to facilitate modification of the educational 
spirit of institutional programmes rather than merely demonstrate that 
change can occur wiUiin one small umt. 

To accomplish this task, it is essential for an innovator to avoid the 
role of a missionary who comes with ready answers to educational 
problems, and to cultivate the role of an inquires who seeks those 
answers with others. There are no universal solutions to be applied 
without regard for the social, cultural, economic, and organizational 
realities of a particular institution. Although general findmgs may be 
useful in a specific institutional context, even more helpful will be the 
systematic assembly of mstitutional data that can replace the intuitive 
thrusts that underlie so much decision-malang in contemporary medical 
education. It is unrealistic to expect that teachers Mdll be persuaded by 
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evidence alone, but without data the opinions that support one view have 
no more substance than those that support another. And where data 
are sparse, opinions are almost mvariably strong, which means that 
irrelevant matters such as individual status or departmental presdge 
rather than substantive information may become decisive influences m 
the process of educational planning. 

The presence of administrative support, faculty mvolvement, and a 
persistent spirit of mquiry does not assure the successful incorporation 
of educational science mto a medical school programme, but their 
absence heightens the probability of failure. The Illinois programme 
was fortunate in having the enthusiastic backing of successive senior 
administrators, a faculty committee structure that allowed leadership 
responsibility to be taken by teachmg staff who increasmgly came to 
value the contributions of professional educators, and a growing 
educational research and development staff who brought to their 
support role not only a high level of technical expertise but also a 
willingness to put their views to the test of study in the medical school 
setting. Yet even these advantages, coupled with sufficient funds to allow 
the work to proceed without major budgetary constraints, did not mean 
quick acceptance. Only after a decade has the widespread resistance 
to such proposals as those described earlier been replaced by grudging 
respect; but there are still recognizable and sometimes vocal pockets of 
resentment because of the significant erosion of departmental autonomy 
that has attended these changes. 

And such feelings are by no means merely manifestations of 
parochial interests. There are some who believe sincerely that dilution 
of departmental dominance in programme planning and decision-making 
has substantially weakened academic quality. Those who regard the 
medical school, like the University of which it is a part, as primarily 
a community of scholars responsible only to that community for the 
content and quality of the programme would probably agree with 
this stand. But others, who see the medical school as an instrument of 
society, not of the faculty alone, would probably note that the 
introduction of the science of education has forced tlie critical scrutiny 
of purpose and procedure that science always demands of its 
practitioners, and has fostered a lystem of accountability based upon 
planned study and assembly of e/idence rather than upon opinion. 

Taken m the perspective of 15 years this work has not sunply 
survived, it has thrived. The original staff of 2 has grown to nearly 50; 
their initial role of observers and reporters has changed to that of 
partners in the development of educational strate^es and tactics; the 
limited educational research programme with which they started has 
grown into a major multi-project enterprise; the informal persuasion 
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of local faculty that there was sometuing" about education worth 
knowing has developed into a teadier-traming effort that even includes 
a formal degree programme and attracts trainees from all parts of the 
world. But taken in the perspecUve of a month, or even a year, progress 
sometimes seems very meagre. Even after so many years of experience, 
discouragement and frustration not infrequendy attend what staff 
members perceive as the snail-like pace of new developments, or the 
distance yet to travel before important goals are fully achieved. Taken 
together, these observations may highlight two qualities beyond technical 
competence that are essential to the successful practitioner of 
educational science in medicine. 

The first is patience. Significant changes occur slowly, and the mote 
significant the change the less swifdy is it likely to occur. No matter 
how expert the impregnator or how eager the carrier, it still takes 9 
months for the united sperm and ovum to evolve into a fully developed 
and functionally secure new bmg. The creation of a new curriculum is 
hard and time-consuming educational work, but it is nothing compared 
with the creation of a new educational spirit that is essentially 
independent of curricular structure. Quick change in the appearance 
of educational programmes is certainly possible and many schools have 
carried it out, only to wonder several years later why things really 
seem very litde different from what they were. To accomplish more 
than face-lifting requires patience, a quality with which too few are 
generously endowed. 

The second is clear vision of an idealized goal without losing sight of 
the starting point that is an equally important referent It is easy to be 
critical of any programme that has obvious flaws and falls short of 
what it has set out to accomplish, and such critique offered as part of a 
feedback loop, rather than as ultimate judgement, can be very helpful. 
But it, is equaUy important to look in the other direction, to determine 
whether what has been achieved represents movement towards the 
long term goal. In heightening aspiration, few things are more effective 
than success, and thus periodic recognition that intermediate goals are 
being achieved is as important as contmued vision of the ideal, which 
may still be very far away. 

The 15-year experience summarized here and the guiding principles 
extracted from it* are neither more nor less than a personal distillate. 
They certainly do not represent a new gospel to be grasped or a new set 
of theories to be tested. They describe what took place in a large, 
conventional, well-established, state-supported medical school in die 
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Usltcd States as it struggled to integrate a new and unfamiliar discipline 
— that of educaton — into its institutional fabric. The experience would 
certainly have been different in a small school or one that was in the 
process of development Although there is a steadUy growing number 
of new schools in the world, many seem to age very quickly as the 
bold ideas with which they were bom harden into new orthodoxy. And 
the problems of large mstitutional size are the subject of comment 
everyrvhcre. For these reasons even such an account as that given here 
may have some practical usefulness to other medical educators. At least 
as they encounter problems and experience discouragement it may bring 
some comfort to know that they are not alone. 
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